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ABSTRACT 


The United States Navy’s Cryptologie Carry-On Program Offiee manages a 
portfolio of Information Warfare (IW) systems. This research and case study 
demonstrate how the Knowledge Value Added (KVA) Methodology can be used to 
formulate a framework for extracting and analyzing performance parameters and 
measures of effectiveness for each system. KVA measures the effectiveness and 
efficiency of CCOP systems and the impact they have on the Intelligence Collection 
Process (ICP) on board U.S. Navy Ships. By analyzing the outputs of the subprocesses 
involved in the ICP in common units of change, a price per unit of output can be 
generated to allocate both cost and revenue at the subprocess level. With this level of 
financial detail, a return on investment (ROI) analysis can be conducted for each process, 
or asset. 
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I. INTRODUCTION 


A. PURPOSE / PROBLEM STATEMENT 

Millions of dollars are spent eaeh year on quiek reaction capability (QRC) 
Information Warfare (IW) and intelligence collection systems. These information 
technology (IT) systems are used to fill existing capability gaps at the Fleet, theater and 
strategic level. Managing these portfolios of equipment as an investment requires that 
program managers be able to maximize the benefits of the systems in a systematic 
manner while considering the impact of cost throughout the enterprise. To successfully 
accomplish this task, it is essential to develop a means and methodology to measure the 
performance of the IT investments in an objective manner. As of yet, no acceptable 
measure of their performance or benefit has been developed to determine whether the 
systems are yielding an adequate level of performance or return on investment (ROI). 

B, BACKGROUND 

Intelligence, as stated in the National Security Strategy (NSS) of the United 
States, is the nation’s first line of defense against terrorists and other hostile nations. The 
document calls for innovation within the armed forces to transform intelligence and 
exploit the country’s advantages in science and technology, stating the following: 

Innovation within the armed forces will rest on experimentation with new 
approaches to warfare, strengthening joint operations, exploiting U.S. 
intelligence advantages, and taking full advantage of science and 
technology [...] We must transform our intelligence capabilities and build 
new ones to keep pace with the nature of these threats, t 

According to the NSS, intelligence capabilities must be transformed and new capabilities 
must be built to keep pace with emerging threats. Intelligence systems must be 
appropriately integrated with defense and law enforcement systems as well as Allied 
systems. The NSS calls for hard choices to “ensure the right level and allocation of 
government spending on national security.”2 It also calls for an investment in future 
intelligence capabilities and a strong security infrastructure to prevent the compromise of 

1 United States. President. The National Seeurity Strategy of the United States of America. 
Washington. White House. 2002 . p. 30. <http://www.whitehouse.gov/nsc/nss.pdf> [1 June, 2005]. 

2 Ibid. pp. 30-31. 
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those capabilities. Lastly, the NSS acknowledges the managerial changes that must be 
made stating that “[w]e must also transform the way the Department of Defense is run, 
especially in financial management.”3 

Transformation of the business practices internal to the Department of Defense 
(DoD), to include financial and program management, is a key enabler for achieving the 
intelligence goals of the NSS.4 With this mandate, the DoD established the Defense 
Acquisition System, DoD Directive 5000.1, dated 12 May 2003, to manage the 
technological investments, resources and programs required to accomplish the NSS and 
support the Armed Forces. The primary objective of the DoD 5000 Series is to acquire 
“quality products that satisfy user needs with measurable improvements to mission 
capability and operational support, in a timely manner, and at a fair and reasonable 
price.” 5 To realign the individual services’ intelligence efforts with the DoD 
transformational practices, coordinative mechanisms such as the Joint Intelligence 
Surveillance and Reconnaissance (ISR) Operational Concept and the Joint ISR 
Operational Architecture (ISR JOC/JOA), have been created.6 

1. Navy ISR 

The Naval Transformation Roadmap of 2003 calls for the reengineering of 
maritime ISR to align with the DoD’s 5000 Series and joint warfighting concepts with the 
goal of redefining standards and metrics and ensuring interoperability, while providing 
the warfighter required capabilities in a timely, cost-effective and efficient manner.7 
Maritime ISR lies at the core of the Naval Operational Doctrine and is an essential 
element in improving the speed and effectiveness of naval and joint operations. It is 
necessary, with today’s threat, to expand the range of ISR options available to the 
commander and ensure decision superiority across the range of military operations in 
accordance with the NSS. From a program management point of view, there are several 

3 United States. President. The National Seeuritv Strategy of the United States of Ameriea. 
Washington. White House. 2002 . p. 30. <http://www.whitehouse.gov/nse/nss.pdf> [1 June, 2005]. 

4 Ibid. 31. 

5 Department of Defense. DoD Direetive 5000.1 . “The Defense Aequisition System.” 12 May 2003. p. 

2 . 

6 Department of the Navy. Naval Transformation Roadmap 2003: Assured Aeeess & Power 
Proieetion.. .From the Sea . Washington: Dept, of the Navy, 2003. pp.68-69. 

7 Ibid. 69. 
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challenges in meeting future requirements. First, program mangers must ereate a 
requirements management program that will thoroughly eapture the joint warfighters’ 
seemingly intangible inputs and translate them as aecurately as possible to ensure the 
system will satisfy the user requirements. Next, needs and requirements must be 
transformed into a comprehensive set of product and process system deseriptions to 
“solve” the ISR problem in a top-down manner by applying the Systems Engineering 
Proeess (SEP). The requirement for the integration of cutting-edge teehnology and 
limited resources must be leveraged with the use of Commercial Off-The Shelf and 
Government Off-The-Shelf (COTS/GOTS) systems in an Open Architeeture (OA) that 
will integrate seamlessly with existing systems. Eastly, innovative teehniques to 
metricize Key Performance Parameters (KPPs) must be developed to facilitate integrated 
Testing and Evaluation of Navy and Joint ISR programs. 

With this backdrop, the goal of Navy ISR program managers is to develop an 
investment strategy for maritime ISR systems eonsistent with validated warfighter 
requirements and within the framework of approved architeetures. The strategy should 
adhere to a national-level strategic view, and balance the development of new 
intelligenee collection capabilities, modernization of current systems, and sustainment of 
existing infrastructures, as well as move toward robust interoperability. The DoD 5000 
Series and the Joint Capabilities Integration and Development System (JCIDS), 
delineated in the Chairman Joint Chiefs of Staff Instruetion (CJCSI) 3I70.0IE, dated II 
May 2003, and other governing doeuments such as the Government Performanee and 
Results Act (GPRA), and the Information Teehnology Management and Results Act 
(ITMRA) serve as a guidelines for addressing these ehallenges. 

2, The Cryptologic Carry-On Program 

The Cryptologic Carry-On Program (CCOP) is a product of the Advanced 
Cryptologie Systems Engineering program, whieh develops state-of-the-art Intelligence, 
Surveillance and Reeonnaissanee (ISR) eapabilities in response to Combatant Command 
requirements for a quick-reaction surface, subsurface and airborne eryptologie earry-on 
capability. CCOP systems have broad scope and funetions. There are approximately 100 
eryptologie capable surface ships in the current Navy inventory. Eaeh of these ships is a 
potential user of earry-on equipment. In addition, there are numerous subsurface and air 
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platforms that are also potential users. CCOP provides the necessary resources to enable 
a rapid transition of available Commercial off the Shelf (COTS) and Government off the 
Shelf (GOTS) technologies that apply ISR system functionalities. These technologies 
typically require various levels of integration to leverage on-board capabilities that 
provide system and mission management, product reporting and data analysis support. 
COTS/GOTS systems usually require some level of adaptation or modification to meet 
fleet requirements. Before deployment for operational use, systems must be 
systematically tested to ensure suitable and reliable operation. They must also be tested 
for network vulnerabilities (if connected to Navy or Joint networks), and tested against 
joint interoperability requirements. 8 

3. Acquisitions Implications 

The Department of Defense Directive 5000.1, 12 May 2003, states that advanced 
technology shall be integrated into producible systems and deployed in the shortest time 
practicable to address “capability gaps” between stated national security threats and the 
ability to properly address them with current capabilities.9 Evolutionary Acquisition 
(EA) is the preferred DoD strategy for rapid acquisition of mature technologies. An 
evolutionary approach delivers capability in increments, recognizing, up front, the need 
for future capability improvements, lo The CCOP shares this objective of EA, which is to 
balance needs and available capabilities with resources, and to put capability into the 
hands of the user quickly. 

a. COTS/GOTS and Open Architectures 

The DoDD 5000.1 also states “A modular, open systems approach 
(MOSA) shall be employed, where feasible.” Historically, cutting-edge ISR systems 
were costly and often “stovepiped.” They were not readily adaptable or reconfigurable 
for a rapidly changing global threat. The DoD mandated use of MOSA and a migration 
to open systems architectures that incorporate commercial solutions to address the fast- 

8 Department of Defense. “The Advaneed Cryptologic Systems Engineering Program: Budget Item 
Justification.” Washington: February,2004. 

<http://www.dtic.mil/descriptivesum/Y2005/Navy/0204574N.pdf#search='Advanced%20Cryptologic%20S 
ystems%20Engineering%20program'> [April 2005] 

9 DoDD 5000.1, 12 May 2003. p. 2. 

10 Department of Defense. DoD Instruction 5000.2 . “Operation of the Defense Acquisition System.” 
12 May 2003. p. 3. 
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paced threat. A number of information systems arehiteetures, polieies, proeedures and 
standards have been developed and implemented to enable the interoperability of these 
new systems and faeilitate the migration, n MOSA enables CCOP systems to be 
designed for affordable change with redueed life eycle eosts. It also facilitates the 
employment of evolutionary aequisition with more opportunities for technology in s ertion 
and shortens the total aequisition time. 

b. Risk Management 

The Risk Management Guide for DoD Aequisition (June 2003) defines 
risk management as “the art and science of panning, assessing, and handling future events 
to ensure favorable outcomes. The alternative to risk management is crisis management.” 
The use of COTS/GOTS and MOSA in the CCOP all provide a level of risk, which must 
be managed. The increasing system teehnologieal complexities, the reliance on complex 
software, along with shortened aequisitions timelines and funding instability all 
contribute to this risk. Modeling and simulation (M&S) tools should be used to mitigate 
some of the risk in the CCOP. In partieular, eost models should be employed to 
determine projected lifecycle costs along with manpower and performance M&S tools. 
The Defense aequisition Guide Book (Nov 2004) suggests that alternatives to the 
traditional cost estimation need to be considered because legacy cost models tend not to 
adequately address costs associated with information systems. 12 
C. RESEARCH OBJECTIVES 

The objeetive of this researeh is to develop a decision support model and 
methodology to assist in the budgeting proeess for the United States Navy’s Chief of 
Naval Operations (OPNAV) CCOP Program Office (OPNAV N201) aequisition of 
information warfare systems. The model will assess the effeetiveness and efficiency of 
the CCOP portfolio of IW systems in support of the achievement of the Navy ISR 
mission. The methodology will be used to develop quantitative objectives that are based 
on measurable data and quantifiable eriteria. Onee measures are defined, an objeetive 
performance baseline for eaeh system in the portfolio ean be ereated with whieh to 

11 Department of Defense. Maritime SIGINT Architecture Technical Standards Handbook v. 1.0 . Ft. 
George G. Meade, MD: NSA/CSS, 23 August 1999. p. 6. 

12 Department of Defense. The Defense Acquisition Guidebook . The Defense Acquisition University. 
Sect. 4.5.7.2 
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compare the impact of existing and future CCOP systems on individual activities, 
proeesses and operations. The resulting information can then be utilized to make sound 
finaneial decisions and projections in the aequisition and deployment of these systems. 

D, METHODOLOGY 

This thesis will attempt to model the intelligence eollection proeess (ICP), as it 
applies to shipbome signals intelligence (SIGINT) eollection, and measure the impact of 
CCOP Systems on the proeess utilizing the Knowledge Value-Added (KVA) 
methodology. First, all assets, subproeesses, and outputs will be identified. The analysis 
of assets will encompass all cost data related to each asset in the proeess, human and IT. 
The subprocess analysis will eontain a detailed breakdown of the ICP to include the time- 
to-leam and number of executions for each subprocess. The proeess outputs will also be 
analyzed to include the total number of proeess outputs and a surrogate revenue stream 
that will be used to monetize the outputs. Next, asset eosts will be allocated throughout 
the subprocesses in whieh they eontribute to the production of outputs. The time-to- 
learn, or knowledge embedded in each subprocess will be multiplied by the number of 
exeeutions of that subprocess. The resulting figure will be used as a basis for the KVA 
Approaeh for alloeating revenue at the subproeess level. Once a cost and benefit for eaeh 
subprocess has been determined, performance ratios such as ROA and ROI can be 
ealculated. 

Comparing performance measures of various eonfigurations of CCOP systems 
within the shipborne ICP can assist program managers in identifying targets of 
transformation, which will aid in making higher-level aequisition strategy decisions. 
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II. RETURN ON INVESTMENT 


A, WHAT IS ROI? 

While there are many forms of “return” sueh as return on equity (ROE), return on 
performanee (ROP) and return on assets (ROA), the foeus of this researeh is the 
overarehing eoneept of Return on Investment (ROI). 

1. ROI Defined 

Return on Investment or ROI has been defined in many ways, but it has generally 
been known to eneompass the quantifieation of the past, present, and potential future 
performanee of an organization or business. More speeifieally, the return on investment 
in assets is a measure of performanee that is determined by the pereentage relationship of 
earnings to assets.13 Many eompanies and organizations view ROI both as a goal as well 
as a measure of profit and asset performanee. In its most generie form, it is eharaeterized 
by the following equation: 14 

„ Earnings 

PercentageROl - - 

Investment 

In this equation, earnings, or the net operating ineome of an organization or business is 
the differenee between revenue and expenses. Investment ean also earry a wide 
eonnotation, but in a general ease, the amount of investment is determined by an 
examination of an organization’s original eapital and its assets, both finaneial and 
physieal. Thus, a higher return, or pereentage ROI, indieates a more effieient and 
effeetive use of those assets and eapital. 

Another view of ROI, by Clarenee Niekerson, a Professor of Business 
Administration Emeritus at the Harvard University Graduate Sehool of Business 
Administration, reeognizes that “the value of a business property is dependent on what it 


13 Nickerson, Clarence B. Accounting Handbook for Nonaccountants . 3 '^^ Ed. New York: Van 
Nostrand Reinhold Co., 1986. p. 73. 

14 Ibid. 632. 
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can produce.”15 He also states, “in order to judge the value of the wealth ereated, we 
should take into aceount the property required to produee it.”i6 

As a praetieal mater, ROI ean be used to determine the potential benefit of a 
single investment strategy or of multiple strategie investment options. Foreeasting 
revenue streams and eomparing them with the expeeted eapital investment and operating 
eosts of a single strategy over the eourse of time, or of multiple strategies ean aceomplish 
this. ROI yields valuable insight for managers and investors making high-level business 
strategy deeisions. But what if an organization produees no measurable “revenue,” in the 
business sense of the word? 

B, ROI IN THE PUBLIC SECTOR 

As the budgets of public sector organizations are eoming under inereased 
scrutiny, stakeholders, managers and taxpayers are all demanding a higher level of 
aeeountability and transpareney of the publie investment in those organizations. 
Additionally, regulations sueh as the Government Performanee Results Aet of 1993 
(GPRA), require the establishment of strategie planning and performanee measurement in 
programs for the aeeountability of their expenditures. Faeed with these challenges, 
public sector organizations have been, for many years, searehing for a defensible means 
of produeing the metries required to measure the value of the serviees and produets they 
provide. In some eases, an organization may produee outputs that are similar to those of 
a eommereial organization or business, allowing that organization to direetly plaee a 
value to its produets and serviees based on the market priee or value of the eommereial 
produets. But many publie seetor organizations are inherently different, and require a 
different approaeh to the determination of their ROI. 

1. The Public Sector Problem 

There are several issues that must be eonsidered when eondueting an ROI analysis 
in the publie seetor. First, the frequent absenee of measurable revenues and profits make 
it a challenge to determine the overall benefit stream produced by the organization. 
Seeondly, hard data is often difficult to collect amidst an abundance of seemingly 

15 Nickerson, Clarence B. Accounting Handbook for Nonaccountants . 3 '^^ Ed. New York: Van 
Nostrand Reinhold Co., 1986. p. 652. 

16 Ibid. 
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intangible, soft data. Also, ROI depends on the notion of eosts and benefits. In the 
publie sector though, the recipients of the benefits or stakeholders are not easily 
identifiable; potential beneficiaries could be program participants, managers of 
participants, sponsors of the program or even taxpayers who are concerned about their 
tax-dollars. Another problem lies at the very nature of public-sector organizations and 
government services in particular, whose services are “essentiaf ’ and must be provided, 
regardless of the accountability or cost. Lastly, public-sector organizations do not often 
have the range of flexibility or options to correct problems and make radical changes. 17 

2, The Public Sector Reality 

The reality of public-sector organizations is that they do provide benefits, 
although in most cases these are presented in the form of cost savings resulting from 
improved quality, increased productivity or timesaving initiatives. Monetizing the 
benefits directly may be difficult, but new approaches to the problem are currently being 
developed. The issue of hard data is also one that can be overcome; output, quality, cost 
and time are all usually measurable in private as well as public-sector organizations and 
represent viable categories of hard data. The notion that services provided by 
government organizations are essential and above scrutiny is also being dispelled; 
regulations such as the GPRA are requiring all agencies to undergo evaluation to ensure 
they are operating effectively and efficiently. 18 And finally, identifying flexibility or 
options in public-sector organizations is not only possible, but required if a desire exists 
to improve the effectiveness of public programs and services. 

This research focuses on public sector ROI as it applies to the Department of 
Defense and a portfolio of information technology (IT) assets. Because of the variable 
nature of IT and their benefits, determining ROI on them poses some unique challenges. 

C. IT ROI 

Organizations, public and private, make investments in information technology 
with the goal of improving the efficiency, timeliness, quality and effectiveness of their 

17 Phillips, Patricia P. “ROI in the Public Sector.” Ed. Patricia Phillips. Alexandria, VA: American 
Society for Training and Development, 2002. <http://www.astd.org/NR/rdonlyres/779B0547-EBlF-48E5- 
8776-7808107FD010/0AVhitepaper.pde/search='ROl%20in%20the%20pubhc%20sector'> [August 2005] 

18 Ibid. 
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services, products or processes. The problem these organizations face though, is 
discerning the amount of value contributed by their IT investments on the processes in 
which they are meant to operate. 

1. Current Approaches 

There have been numerous approaches to assessing the impact of IT on an 
organization’s fiscal performance at the corporate and sub-corporate levels of 
aggregation. Table 1 summarizes some of the most common approaches to measuring 
the return on IT and the advantages and limitations of each. 


Level of 
Analysis 

Approach 

Focus 

Example 

Key 

Assumption 

Key 

Advantage 

Limitation 


Process of 
Elimination 

Treats effect of 
IT on ROI as a 
residual after 
accounting for 
other capital 
investments 

Knowledge 

capital 

ROI difficult 
to measure 
directly 

Uses 

commonly 

accepted 

financial 

analysis 

techniques 

and existing 

accounting 

data 

Cannot drill down to 
effects of specific IT 
initiatives 

Aggregate 
Corporate 
(firm) level 

Production 

Theory 

Determines the 
effects of IT 
through input 
output analysis 
using regression 
modeling 
techniques 


Economic 
production 
function 
links IT 
investment 
input to 
productivity 
output 

Uses 

econometric 
analysis on 
large data 
sets to show 
contributions 
of IT at firm 
level 

"Black-box" approach 
with no intermediate 
mapping of IT's 
contributions to outputs 


Resource- 

Based 

View 

Linking firm 
core 

capabilities 

with 

competitiveness 


Uniqueness 
of IT 

resource = 

competitive 

advantage 

Strategic 
advantage 
approach to 
IT impacts 

Causal mapping between 
IT investment and firm 
competitive advantage 
difficult to establish 

Corporate/ 

sub¬ 

corporate 

Option 

Pricing 

Model 

Determines the 
best point at 
which to 
exercise an 
option to invest 
in IT 


Timing 
exercise 
option = 
value 

Predicting 
the future 
value of an 

IT 

investment 

No surrogate for revenue 
at sub- corporate level 
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Family of 
Measures 

Measure 
multiple 
indieators to 
derive unique 
eontributions of 
IT at sub- 
eorporate level 

Balanced 

score-card 

Need 
multiple 
indicators to 

measure 

performance 

Captures 
complexity 
of corporate 
performance 

No common unit of 
analysis/ theoretical 
framework 

Sub¬ 

corporate 

(Process) 

Cost-Based 

Use eost to 
determine value 
of information 
teehnology 

Activity- 

based 

costing 

Derivations 
of cost = 
value 

Captures 
accurate cost 
of IT 

No surrogate for revenue 
at sub- corporate level — 
no ratio analysis 


Knowledge 

Value 

Added 

Alloeating 
revenue to IT in 
proportion to 
contributions to 
process outputs 

Housel & 
Kanevksy 
KVA 

Methodology 

IT 

contributions 
to output = 

IT value- 
added 

Allocates 
revenue and 
cost of IT 
allowing 
ratio analysis 
of IT value- 
added 

Does not apply directly 
to highly creative 
processes 


Table 1. Common Approaches to Measuring Return on IT. 19 


These approaches are subdivided into two distinct levels of analysis, the corporate level 
and the sub-corporate level. Corporate-level approaches seek to determine the 
contribution of the assets, both knowledge and IT, on the overall performance of the 
organization. The sub-corporate-level approaches look internally at the sub-processes 
which are involved in the production of organizational output and attempt to establish a 
measure for the benefits of knowledge and IT assets within each sub-process. The goal 
of each approach is to provide managers with a metric of value for their organization’s 
investment in IT and knowledge assets. 

D, MANAGING IT INVESTMENTS IN THE DOD 

It is the stated goal of the Department of Defense (DoD) that all information 
technology (IT) programs within the organization be managed as investments rather than 
acquisitions. To accomplish this, and to fall within the framework of regulations and 
legislation such as the Government Performance and Results Act (GPRA) and the 
Information Technology Management and Results Act (ITMRA), the DoD has made it its 
goal to establish performance measures in the IT investment process. 


19 Housel, T., Jansen, E., Pavlou, P. & Rodgers, W. “Measuring the Return on Information 
Teehnology: A Knowledge-Based Approaeh for Revenue Alloeation at the Proeess and Firm Level.” 
Monterey, CA: Naval Postgraduate Sehool, August 2005. 
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1, Defining IT Performance Measures in the DoD 

The Department of Defense Guide for Managing Information Teehnology (IT) as 
an Investment and Measuring Performanee v.I.O (10 February 1997), defines IT 
performanee measure as: 

the assessment of effectiveness and efficiency of IT in support of the 
achievement of an organization’s missions, goals, and quantitative 
objectives through the application of outcome-based, measurable, and 
quantifiable criteria, compared against an established baseline, to 
activities, operations, and processes [...] Performance Measurement is the 
means by which an organization measures it effectiveness and efficiency 
in the pursuit of its missions, goals, and objectives .20 

Effectiveness implies accomplishing the required mission tasks and goals while ensuring 
customer satisfaction and high quality. Efficiency implies that the best possible use of 
available resources was employed to improve quantity or work, reduce costs and improve 
timeliness of the overall project. 

2, Steps for Effective Performance Measurement 

To assist managers in maximizing the benefits of IT investments, the guide 
outlines six steps in effective performance measurement:2i 

• Define mission, key result areas, and business functions. 

• Develop mission related goals. 

• Generate performance measures/indicators. 

• Validate and verify performance measures. 

• Implement the performance measures and collect data. 

• Monitor and assess the results and repeat the process as needed. 

3, Levels of Management 

The DoD also recognizes that managers at different levels of any organization 
require different types of information from which to base their investment and business 

20 Department of Defense. Guide for Managing Information Technology (IT) as an Investment and 
Measuring Performance . Version 1.0. Arlington, Virginia: Vector Research Inc., 10 February 1997. ES-i. 

21 Ibid. 
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strategies on. The guide identities three management levels: Enterprise, Funetional, and 
Program/Projeet Level .22 At the Enterprise Eevel, managers are focused on policy and 
mission decisions to guide the strategic direction of acquisition decisions. The 
Functional Eevel managers focus on unit results and improving the operational processes 
which produce them, mission related outcome measures are also developed at this level. 
At the Program/Project Eevel, managers and project leaders focus on the activities and 
tasks critical to making and executing tactical and management decisions. Program 
managers and project leaders are also the focal point from which the other levels receive 
detailed performance and valuation data for individual systems, projects, processes and 
activities. This is the level where the CCOP Program Office functions and where this 
research was primarily focused. 

4, Performance Measurement at the Program/Project Level 

Program/Project-Eevel managers and leaders capture and generate the information 
required by managers at the other two levels. It is necessary, at this level, to determine 
what key information is needed to develop defensible, accurate and meaningful 
performance measures. Among these measures is Return on Investment, ROT 

At a minimum, ROI consists of both a return (i.e., benefit) component and an 
investment (i.e., cost) component. The cost component measures the number of 
investment dollars management has placed into the human, infrastructure and IT assets of 
the program or project; this information is generally documented in detail. Benefit 
measures are much more complex. The DoD guide defines “the accomplishment of 
functional missions and goals,”23 as a measure of benefit. How a program, project, 
process or asset contributes to overall mission accomplishment is normally a matter of 
efficiency and effectiveness. The question often raised though is “how is efficiency and 
effectiveness measured and how do they translate into dollars?” 

To be mathematically sound, ROI requires that benefits AND costs be monetized. 
While mangers or analysts can directly measure the benefits of some investments in 
monetary terms (e.g., cost savings or lower overhead), oftentimes they cannot be directly 

22 Department of Defense. Guide for Managing Information Technology (IT) as an Investment and 
Measuring Performance . Version 1.0. Arlington, Virginia: Vector Research Inc., 10 February 1997. p. 3-4. 

23 Ibid. 6-9. 
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measured monetarily. Sueh is usually the ease with many IT systems, projeets and 
programs. The DoD Guide summarizes 12 approaehes to measuring the performanee of 
IT that are similar to those in Table 1, but recognizes that the list is not complete.24 The 
following chapters outline an alternative method and approach. 


24 Department of Defense. Guide for Managing Information Technology (IT) as an Investment and 
Measuring Performance . Version 1.0. Arlington, Virginia: Vector Research Inc., 10 February 1997. p. 8-1. 
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III. THE KNOWLEDGE VALUE ADDED METHODOLOGY 


A, THE COMPARABILITY AND VALUE PROBLEM 

There are several situations the realm of business and non-profit organizations 
that can benefit from a method for measuring the value of knowledge assets both human 
and IT. The first hurdle in solving the problem is defining what is meant by “value”. 
The simplest measure of value, as presented by Dr. Myron L. Cramer, is dollars: how 
much is earned or saved. The metric of dollar-value is primarily used in assessing 
contributions in a commercial or business context.25 A problem exists though, when 
applying dollar-value in the DoD case where different methods are used to assess how 
systems or people contribute “value” to combat effectiveness or mission completion. 
These methods often fail to capture the benefit stream produced by organizations, 
processes or assets in standard or comparable units of measure. And oftentimes, the only 
metric that is measured with any level of standardization and accuracy is cost. Benefits, 
while measured in terms of varying degrees of contribution, are seldom represented in the 
monetized fashion required by traditional accounting methods for extracting a value or 
Return on Investment. 

Another problem present in the DoD case is the translation of outputs into monetary 
benefits. Whereas in the commercial case, a price per unit is assigned to the outputs, 
there is no equivalent pricing mechanism in the DoD or non-profit case. This presents a 
problem when conducting empirical financial analysis and in particular when seeking a 
baseline from which to formulate sound fiscal decisions. 

B, THE KVA SOLUTION 

1, Knowledge Value-Added (KVA) Theory 

Knowledge Value-Added (KVA) created by Dr. Tom Housel (Naval Postgraduate 
School) and Dr. Valery Kanevsky (Agilent Labs). Initiated almost 15 years ago, KVA 
was built on tenets of complexity and entropy theory, and was developed in response to 
business process reengineering efforts of organizations that, with no other method 

25 Cramer, Myron. “Measuring the Value of Information.” Atlanta, GA: Georgia Teeh Researeh 
Institute, <http://iw.gtri.gateeh.edu> [20 July 2005]. p. 2 
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available, focused primarily on cost cutting measures to determine the “value” of a 
process or knowledge asset. KVA is based on the assumption that humans and 
technology in organizations add value by taking inputs and changing them into outputs 
through core processes.26 The amount of change an asset or process produces can, in 
fact, be a measure of value or benefit. 

KVA is a general theory for estimating the value added by knowledge assets, 
human and IT, using a methodology that is analytic and tautological. It is based on the 
premise that businesses and other organizations produce outputs (e.g., products and 
services) through a series of processes and subprocesses which change, in some manner, 
the raw inputs (i.e., labor into services, information into reports). KVA explains the 
changes made on the inputs by organizational processes to produce outputs in terms of 
the equivalent corresponding changes in entropy. The concept of entropy is defined in 
the American Heritage Dictionary as a “measure of the degree of disorder [or change] in 
a closed system.” In the business context, it can be used as a surrogate for the amount of 
changes that a process makes to inputs to produce the resulting outputs. 27 

2. KVA Assumptions 

KVA uses knowledge-based metaphor to operationalize the relationship between 
change in entropy and value added. The units of change induced by a process to produce 
an output are described in terms of the knowledge required to make the changes. More 
specifically, the time it takes the average learner “to acquire the procedural knowledge 
required to produce a process output provides a practical surrogate for the corresponding 
changes in entropy.”28 This concept is illustrated in Figure 1. 


26 Housel, T. and Bell, A. Measuring and Managing Knowledge . Boston: McGraw-Hill, 2001. pp. 92- 
93. 

27 Housel, T. El Sawy, O., Zhong, J., and Rodgers, W. “Models for Mearsuring the Reutm on 
Information Technology: A Proof of Concept Demonstration.” 22"“^ International Conference on Inormation 
Systems. December, 2001. p. 13. 

28 Ibid. 
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;■ P(.x) = V 


Process /* is a business process. 

If input X is equai to output^, no vaiue has been added by process P. 

Ifinputxis changed by process Pinto outputvaiue has been added and iscc change. 
Change can be measured by the amount ofknowiedge required to make the change. 

This knowiedge«the amount of time ittakesforan average learnerto acquire the knowiedge. 
So, vaiue added by process « change fs knowiedge required to produce the outputsss K 


Figure 1. The FIousel-Kanevsky Value-Added Cycle.29 


KVA theory also assumes that all process outputs can be described in equivalent 
units (e.g., the time required to learn how to produce outputs). The advantages of this 
level of process output description include:30 

• The ability to compare all processes in terms of their relative productivity 

• The ability to allocate revenue to a common unit of output 

• The ability to describe the value added by IT in terms of the outputs it 
produces 

• The ability to relate outputs to the cost to produce those outputs in 
common units 

• A common unit of measure for organizational productivity 

By describing all process outputs in common units it is possible for managers and 
analysts to assign revenue, as well as cost, to those units at any given point in time. With 
the resulting information, traditional accounting and financial performance and 
profitability metrics can be applied at the sub-organizational level. 

Other assumptions include the concept of Learning Time. Learning Time is 
measured in common units of time and is a surrogate for the procedural knowledge 
required to produce process outputs. Units of Learning Time represent common units of 
output which are described by the variable K. Procedural knowledge can be embedded in 
process assets such as IT, employees, training manuals, etc. A single execution of a 
process is equivalent to a single unit of output, represented by a given number of 


29 Housel and Bell. 2001. 

30 Housel, et al. Deeember 2001. p. 11. 
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common units, K. Additional levels of detail in process descriptions provide additional 
levels of accuracy in the estimation of the number of knowledge units comprising those 
processes. 

3, Three Approaches to KVA 

In addition to the Learning Time approach for calculating KVA, Table 2 
illustrates two other methods which can be used. 


Learning Time 

Approach 

Process Description 
Approach 

Binary Query Method 

Identify compound 
process and its 
component processes. 

1 dentify compound 
process and its 
component processes. 

Identify compound 
process and its 
component processes. 

Establish common units to 
measure learning time. 

Describe the products in 
terms of the instructions 
required to reproduce 
them and select unit of 
process description 

Create a set of binary 
YES/NO questions such 
that all possible outputs 
are represented as a 
sequence of YES/NO 
answers. 

Calculate learning time to 
execute each component 
process. 

Calculate number of 
process description words, 
pages in manual, lines of 
computer code pertaining 
to each process. 

Calculate length of 
sequence of YES/NO 
answers for each 
component processes. 

Designate sampling time 
period long enough to 
capture a representative 
sample of the compound 
processes' final 
product/service output. 

Designate sampling time 
period long enough to 
capture a representative 
sample of the compound 
processes' final 
product/service output. 

Designate sampling time 
period long enough to 
capture a representative 
sample of the compound 
processes' final 
product/service output. 

Multiply the learning time 
for each component 
process by the number of 
times the component 
executes during sample 
period. 

Multiply the number of 
process words used to 
describe each component 
process by the number of 
times the component 
executes during sample 
period. 

Multiply the length of the 
YES/NO string for each 
component process by the 
number of times this 
component executes 
during sample period. 

Allocate revenue to 
component processes in 
proportion to the 
quantities generated by 
previous step. 

Allocate revenue to 
component processes in 
proportion to the 
quantities generated by 
previous step. 

Allocate revenue to 
component processes in 
proportion to the 
quantities generated by 
previous step. 
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Calculate the cost to 
execute each component 
process, calculate return 
on investment per process 
by dividing revenue 
allocated to component 
process by cost of 
component process. 


Calculate the cost to 
execute each component 
process, calculate return 
on investment per process 
by dividing revenue 
allocated to component 
process by cost of 
component process. 


Calculate the cost to 
execute each component 
process, calculate return 
on investment per process 
by dividing revenue 
allocated to component 
process by cost of 
component process. 


Table 2. Three Approaches of KVA.31 


The procedural steps of KVA as applied in this research are outlined in the following 
case study. 


31 Walsh, David. “Knowledge Value Added: Assessing both Fixed and Variable Value.” Business 
Process Audits.Com. White Papers. Business Process Audits.Com, 13 August 1998. 
<http://www.businessprocessaudits.com/kvawalsh.com> [06 June 2005] 
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IV. METHODOLOGY CASE STUDY: USS READINESS 


A. INTRODUCTION 

In late 2004, the staff of the United States Navy’s Chief of Naval Operations 
(OPNAV) direeted its Cryptologie Carry-On Program Offiee (OPNAV N201) to focus on 
three goals for the following fiscal year: Efficiencies, Metrics, and Return on 
Investment.32 Measuring efficiencies, developing proper metrics, and determining return 
on investment of the CCOP portfolio of Information Warfare (IW) systems had, for many 
years, seemed a bridge too far. Faced with this challenge and with the yearly challenge 
of deciding how to allocate precious funding dollars to the twelve IW systems which 
currently make up the CCOP portfolio. Lieutenant Commander Brian Prevo, the CCOP 
program officer had to answer some fundamental questions. Should he merely give each 
an equal amount of continuous funding? Should he ask which ones needed the most 
funding to continue or upgrade? Should he ask the users which ones they liked? None of 
these questions really went to the heart of the problem from his perspective. He really 
wanted a relatively objective estimate of the contribution of performance of each. The 
simple question for him to answer before he made his next moves was, “What is the 
return on investment of each CCOP system?” Further, he wanted to know the best 
strategic “options” for the employment and deployment of his systems. He wanted to 
know where his investments might provide the best future performance in supporting the 
Navy’s Global Intelligence, Surveillance, and Reconnaissance (ISR) mission. 

LCDR Prevo turned to a team from the Naval Postgraduate School’s (NPS) 
Graduate School of Operations and Information Sciences, Dr.’s Tom Housel and 
Johnathan Mun, to help him devise a way to allocate his limited funding resources. He 
also enlisted the help of a fellow Naval Cryptologist and Information Warfare Officer 
who happened to be a student at NPS, LCDR Cesar Rios. Rios had operated CCOP 
systems and other IW systems while conducting ISR missions from various Navy 
platforms, to include ships and aircraft. As team lead and subject matter expert, LCDR 
Rios sought advice from Dr.’s Housel and Mun about how to structure this problem in a 

32 Department of the Navy. CCOP Program Briefing . Power Point. Washington: Dept, of the Navy, 
CCOP Program Office (OPNAV N201C), 25 April 2005. 
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way that would provide the ROI and options information that the CCOP Program Offiee 
needed to make resouree alloeation deeisions. Mun would provide the real options (RO) 
analysis approaeh whieh would be conducted in follow-on to this research. The 
Knowledge Value Added (KVA) approach provides the raw data necessary for this kind 
of analysis. Housel is an expert in the KVA methodology. KVA provides a method for 
describing the output of all process assets (human and information technology) in 
common units. Market comparables can then be generated from the commercial world to 
estimate the price per common unit, which allows revenue estimates for the process 
outputs of non-profit organizations such as the U.S. Navy. It also allows a cost per unit 
of output that, when combined with the revenue estimates, provides the raw data for ROI 
analysis. 

Armed with these methodologies, the team began the process of collecting the 
data required to provide LCDR Prevo with the analysis he needed to support is CCOP 
resource allocation decisions. The following is a synopsis of the research, including the 
use of new software that provides KVA and RO analysis. 

1. Objective 

The overall objective of the research was to develop a model and methodology to 
assist in the CCOP Program Office’s budgeting process for the acquisition of information 
warfare systems. The model was to assess the effectiveness and efficiency of the systems 
in support of the achievement of the Navy Intelligence, Surveillance, and Reconnaissance 
(ISR) mission. The methodology had to be capable of producing measurable objectives 
that were reliable and based on clearly defined criteria. Once measures were defined, an 
objective performance baseline for each system in the portfolio could be created with 
which to compare the impact of existing and future CCOP systems on ISR activities, 
processes and operations. The resulting information could then be utilized to make 
grounded and defensible financial decisions and projections in the acquisition and 
deployment of these systems. 

2, Method 

This case study demonstrated the operationalization of the Knowledge Value 
Added (KVA) Methodology’s ability to extract the measurable and quantifiable data 
required to develop a performance baseline for valuing the impact of CCOP systems on 
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the process in which they are employed, the Intelligence Collection Process (ICP). In 
particular, this case focused on the shipbome ICP and the performance of the human and 
information technology (IT) components that comprise the process. 

B, HYPOTHESIS 

The Performance metrics of CCOP systems operating within the ICP can be 
derived by applying the Knowledge Value Added Methodology. These metrics can be 
used to apply financial estimates for the determination of Return on Investment, Return 
on Assets and Return on Performance. 

C. ANALYSIS AND DATA COLLECTION 

1. The ICP and CCOP 

To accurately measure the performance of CCOP systems it is necessary to 
observe the systems as they operate within the overall ICP along with the fixed 
infrastructure and human operators. Although the ICP differs slightly between the 
multiple collection platforms (i.e., aircraft, destroyers, cruisers, etc.), the process will be 
described at a general-enough level to achieve an aggregate view of the process elements 
and their outputs. 

CCOP systems have a broad scope of capabilities and functions. Within the ICP, 
they may be used to perform operations in a single subprocess or in multiple 
subprocesses simultaneously. CCOP systems may be used in a stand-alone manner, or 
they may augment existing infrastructure or capabilities onboard the ISR platform. The 
systems are also capable of performing fully automated functions as well as human- 
enabled functions. It is this particular mix of dynamic behaviors that makes analyzing 
CCOP systems a challenge. 

2. The Data Collection Challenge 

The ISR Mission is generally conducted at a highly classified level. Many 
particulars of the ICP and the CCOP systems and their outputs are closely guarded 
information. Deriving the information required to perform analysis at the unclassified 
level is difficult, and at times, impossible. For the purpose of academic research, much 
of the data was estimated or inferred based on realistic sampling of unclassified process 
information. Information on human capital, such as salaries and operator training, are 
public information and were gathered from sources such as the Stay Navy Website and 
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the Center for Information Dominanee (CID) training doeumentation. The equipment 
data was also derived or inferred from doeumentation provided by the OPNAV N20 staff 
and the Spaee and Naval Warfare Command (SPAWAR). Other information such as 
number of process outputs and executions were extrapolated from samples gathered via 
interviews with ISR crews currently operating onboard, deployed U.S. Navy surface 
ships. 

D, MODELING THE USS READINESS ICP 

The intelligence collection process (ICP) is the means by which tactical Navy 
intelligence, surveillance and reconnaissance (ISR) units (i.e., ships, aircraft and other 
platforms) receive requests for intelligence or intelligence collection tasking and apply 
various human disciplines and IT technologies to search, acquire, process and report 
results back to tactical users (i.e., fleet staffs and strike groups), and national-level 
consumers (i.e., NSA). The following table is a brief description of the generalized 


process: 


■ 

Sub-Process Name 

Sub-Process 

Description 

P1 

Review Request 

• Determine if coiiection capabiiity is avaiiabie 

• Determine if further direction or info required 

P2 

Determine Op/Equip Mix 

• Review directives and target information to 
determine type/category of target 

P3 

input Search/Function into CCOP 

• Assign search biocks and aiiocate system 
resources to each target 

P4 

Search/Coiiection Process 

• Targeted or fuii spectrum search 

• Observe sensor data for target cues 

P5 

Target Data Acquisition/Capture 

• Audio Routing 

• Record/Capture Data 

P6 

Target Data Processing 

• Demoduiate, decrypt, direction find (DF), or 
Geo-iocate 

• Transiate 

P7 

Target Data Anaiysis 

• Human or iT-based anaiysis of captured data 

P8 

Format Data for Report 

Generation 

• input data into required reporting formats 

P9 

QC Report 

• Check format, accuracy and adherence to 
tasking, reguiations and iaws 

P10 

Transmit Report 

• Transmit via secure voice radio, secure 
internet reiay chat, US Message Traffic 

Format 


Table 3. The Intelligence Collection Process (ICP). 
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Each sub-process can be further broken down into individual actions that may be 
required to perform the subprocess. Below is the breakdown of sub-proeess P6 Target 
Data Processing: 


P6 

Target Data Processing 


Human-based (no automation required) 


Manuai copy directiy into report 


Human transiation & processing 


iT-based 


Direct transfer into report 


Demoduiate 


Aii iT-based 


Human-enabied 


Decrypt 


Aii iT-based 


Human-enabied 


Direction finding 


Automatic - Locai Line Of Bearing (LOB) 


Human-enabied - iocai LOB 


Human-enabied - B-rep request 


Geoiocation 


Speciai processing 


Table 4. Proeess P6 Aetivities. 


1. USS READINESS 

USS READINESS is a fictitious El.S. Navy warship which has been outfitted for 
eondueting ISR missions. Along with the general manning, the ship has a eontingent of 
Information Warfare operators who perform the ICP utilizing CCOP systems that 
intereonneet with a pre-existing IT infrastructure. Eor the purpose of this research, it is 
assumed that the ship is on a typieal six-month deployment and reeeives daily tasking for 
ISR eollection from higher authorities as well as intelligenee requirements from the 
commanding officer and group commanders. In response to tasking, the ISR erew 
onboard USS READINESS produees a variety of reports which include raw-intelligence 
reports, teehnieal reports, analyst-to-analyst exehanges and daily collection summaries. 
The table below illustrates the ISR erew and the subproeesses within the ICP in whieh 
they perform: 
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IW Operator 

Assigned to iCP 
Processes 

Division Officer 

1,2,9 

Division Leading Petty 

Officer 

1,2,9,7 

Signais Operator 1 

3-7,9 

Signais Operator 2 

3-7 

Signais Operator 3 

4-7 

Signais Operator 4 

4-7 

Signais Operator 5 

4-7 

Comms Operator 1 

8,10 

Comms Operator 2 

8,10 

Comms Operator 3 

8,10 

Table 5. USS READIN 

ESS ISR Crew. 


USS READINESS is outfitted with four CCOP systems (A, B, C, and D) which 
operate in the following subprocesses: 



Subprocess Name 

CCOP 

Assigned 

PI 

Review Request/Tasking 

A 

P2 

Determine Op/Equip Mix 

A 

P3 

input Search Function/Coveraqe Pian 

A 

P4 

Search/Coiiection Process 

A,B 

P5 

Target Data Acquisition/Capture 

A,B 

P6 

Target Data Processing 

A,B,C,D 

P7 

Target Data Anaiysis 

A.C.D 

P8 

Format Data for Report Generation 

A 

P9 

QC Report 

A 

P10 

Transmit Report 

A 


Table 6. USS READINESS CCOP Systems. 


As shown in Table 6, CCOP systems may be used in a single subprocess or across 
multiple subprocesses along with the existing infrastructure available in each particular 
platform. Additionally, some systems such as CCOP A are highly complex and 
comprised of multiple subsystems. Below is a breakdown of CCOP A and its related 
components. 
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CCOP A (Example) 

Component 

Description/Functions 

Radio Frequency Management System 

• RF management 

Signal Acquisition System 

• Energy Search 

Audio Distribution System 

• Audio Routing & Recording 

Intermediate Frequency Signal Processing 

System 

• Spectrum Display Operations 

• Signal Processing Applications 

Control & Processing System 

• Coverage Plan Creation/Management 

Common Cryptologic Workstation (CCWS) 

• Database Operations 

• JMCIS Applications 

• Cryptologic Unified Build Applications 

• Microsoft Applications 

• Signal Processing Applications 


Table 7. CCOP A Components. 


To aeeurately eapture the eontribution of CCOP systems aeross subproeesses it is 
neeessary to break down eaeh system to its individual eomponents and funetions, analyze 
the attributes of eaeh, and insert them into the appropriate subproeess of the ICP. 

E. APPLYING KVA 

As deseribed in the previous ehapter, KVA uses a knowledge-based metaphor as a 
means to deseribe units of ehange in terms of the knowledge required to make the 
ehanges. The analysis is aeeomplished by extraeting historieal data to build metries that 
are struetured in the same manner as eommon finaneial metries. The differenee between 
KVA and other finaneial models though, is that KVA provides the means to analyze the 
metries at a sub-organizational level and allows for the alloeation of eost and revenue 
aeross subproeesses for aoeounting purposes. 

1, KVA Assumptions 

The underlying assumptions of KVA are that: 

• Human and teehnology in organizations take inputs and change them into 
outputs through core processes 

• By describing all process outputs in common units (i.e., the knowledge 
required to produce the outputs) it is possible to assign revenue, as well as 
cost, to those processes at any given point in time. 

• All outputs can be described in terms of the time required to learn how to 
produce them. 

• Learning Time is measured in common units of time and is also a 
surrogate for knowledge. Thus, units of Learning Time can also be called 
Common Units of Output (K). 
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• Having a common unit of output makes it possible to compare all outputs 
in terms of eost per unit as well as price per unit, sinee revenue ean now 
be assigned at the sub-organizational level. 

• Once eost and revenue stream have been assigned to sub-organizational 
outputs, normal aeeounting and finaneial performance and profitability 
metries can be applied to them. 

2. Case Study Assumptions and Data 

The following assumptions and data apply to the USS READINESS - KVA proof 
of concept ease study. Figures and results are estimates only, and should not be viewed 
as projeetions of aetual eapabilities. 

a. Assumptions 

Length of Sample Period: The sample period for this analysis was one 
deployment period (6 months). For this reason, some annual eost data is adjusted to 
refieet the sample period. 

Cost Assumptions : Cost of human capital assets was derived from annual 
U.S. Navy salary information available on the Stay Navy website for Fiscal Year 2005 
with allowanees caleulated for a erew home ported in San Diego, CA. Operator rank and 
bonus level information was estimated based on a typieal shipbome ISR crew. 
Equipment eosts were derived from annual cost data provided by the OPNAV N20 staff. 
While equipment eosts refieet installation and training eosts, they do not refieet 
amortization or the total operational eost of the program. 

Proxy Revenue Assumptions: Proxy revenues are based on the following 
assumption: if a eommereial entity or organization produees eomparable outputs as a 
not-for-profit organization sueh as the Department of Defense, and the proeesses required 
to produee those outputs are comparable, eertain inferenees ean be derived. First, if the 
proeesses are eomparable, the outputs of the commercial case are comparable to the 
outputs of the not-for-profit ease. Second, if market forces have placed a “value” or 
price-per-unit to the comparable eommereial outputs yielding a revenue stream for the 
eommereial entity, that priee-per-unit ean also be applied to the not-for-profit ease. 
Fastly, the derived price-per-unit ean be used to develop an analytical or hypothetical 
revenue stream for the not-for-profit organization. For this researeh, the priee-per-report 
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of nine competitive/business intelligence organizations and private military corporations 
were averaged to yield a notional market comparable price-per-unit of the outputs of the 
USS READINESS ICP. Eigure 2 illustrates two of the corporations sampled in the 
Intelligence Price-per-Unit study. 


IHL Consulting Group 

IHL Consulting Group is a global research and advisory firm specializing in 
technologies for the retail and hospitality industries. The company, generates 
timely data reports, offers advisory services and serves as the leading retail 
technology spokesperson for industry and vendor events. IHL provides customized 
business intelligence for retailers and retail technology vendors, with particular 
expertise in supply chain and store level systems. Their customers are retailers 
and retail technology providers who want to better understand what is going on in 
the overall technology market, or wish to identify specific equipment needs for the 
retail market. 

IHL’s price per report ranges from $1,495 to $3,295 

J.C. Owens Global Consulting, LLC 

J.C. Owens Global Consulting, LLC is a subsidiary of J. C. Owens Group 
Worldwide. It has been established to provide international investigation, business 
intelligence and risk consulting services to corporate organizations and government 
agencies worldwide in 182 countries around the world. Their specific areas of capability 
include: Global Corporate Investigations; Global Business Intelligence; Intellectual 
Property/Copyright Trademark Investigations; Background/Pre-employment/Due 
Diligence/Litigation Support; Insurance Fraud & Claim Investigation. 

The firm presently covers a total of 182 countries in Africa, North/South 
America, Middle East, Central America, Europe, Asia, and the Caribbean. It maintains 
an office in Bloomfield, New Jersey, United States of America, from where it covers the 
world; and an office in Lagos, Nigeria, which handles its African operations. 

Price per Giobai inteiiigence Report/Assignment is approximateiy $5,000 (US) 


Figure 2. Intelligence Price-per-Unit Benchmarking Sample.33 

Output Assumptions: Although the ICP onboard the ISR unit produces a 
variety of reports, only raw-intelligence reports (also known as Klieghlights or KL) were 
counted as outputs for this research. It was also assumed that each report was of equal 
tactical value/weight and importance, in essence, all outputs were similar. 

Other Assumptions: IT Learning Time. It was assumed that the 
knowledge embedded in information technology (IT) systems can be derived by 
averaging the time it would take an average learner to learn how to produce the outputs 

produced by the IT systems in a single subprocess output cycle. CCOP systems are 

33 Data for IHL Consulting Group was gathered from the 2005 IGL Consulting Group Research Price 
List which is available from www.ihlservices.com 

Data for J.C. Owens Global Consulting, LLC was furnished by Mr. Israel Mbachu, CFE, CII, Principal 
Partner at J.C. Owens Global Consulting, LLC. Email dated 09 September 2005. 
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highly complex and at times, comprise multiple components with varying functions. To 
estimate the time to learn of a single CCOP system, the components were analyzed 
individually. Academic authorities on the functions performed by each were consulted to 
determine the length of time it would take an average learner (assuming at least a 
Bachelor’s of Science (B.S.) degree in Electrical Engineering or in a field related to the 
component) to learn how to produce the IT outputs. In this case, subject matter experts in 
the functional fields of each system were consulted to estimate the IT time to learn. 
Eigure 3 illustrates one such interview. 


CCOP C Learning Time Derivation Example 

To determine the learning time of CCOP C, the team first dissected the system into its basic 
functional components. CCOP C is the AN/SSO-120(V) Transportable-Radio Direction Finder 
(T-RDF). T-RDF provides a low-cost Medium/High/Very High/Ultra High Frequency 
(MF/HF/VHF/UHF) Direction Finding (DF) capability to selected U.S. Navy ships. T-RDF has 
two major components, the receiving equipment and the processing unit. 

To analyze the system and determine its time to learn, the team consulted Dr. Richard Adler, 
an authority on signals intelligence (SIGINT) systems and antenna technologies. It was 
assumed that, as a baseline, the “average learner” to be taught the functions of T-RDF would 
have an undergraduate degree in a related technical field such as Electrical Engineering. 

Dr. Adler suggested that the underlying disciplines that would have to be learned are: 

-Basic RF Theory (66 days) 

-EM Theory/Formal EM (198 days) 

-Basic Communications Theory (132 days) 

-Propagation Theory (66 days) 

-Antenna Theory (66 days) 

-Basic Radio Direction Finding (66 days) 

Aggregating the results, an estimate of 594 days of learning time would be required for the 
average learner to learn how to produce the outputs of CCOP C. 


Figure 3. CCOP C Eeaming Time.34 35 


34 Department of the Navy. Vision.. .Presenee.. .Power: A Program Guide to the U.S. Navy - 2002 
Edition . Washington: Dept, of the Navy, 2002. Chapter 3. 

35 Dr. Riehard Adler is a Research Associate Professor in Department of Electrical and Computer 
Engineering at the Naval Postgraduate School. He also holds positions in the Research Committee and is 
the Supervisor of the Signal Enhancement Lab. Dr. Adler has 31 years of experience in undergraduate and 
graduate teaching and thesis advising, 29 years in design and analysis of VLF-UHF tactical, strategic, DF 
and broadcast antennas, 31 years in EM numerical analysis of the effects of platforms and environment on 
the performance of antennas, and 26 years Hands-On-Workshops on Numerical Antenna Modeling for wire 
antennas, reflector antennas and general scattering shapes. He is a Registered Professional Engineer in 
California. 
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b. Human Capital Data 

Individual data sheets were created for each member of the fictitious USS 
READINESS ISR crew. The information included the typical rank and subspecialty 
designators or Navy Education Codes (NEC) of afloat-IW operators. Military pay was 
collected along with NEC-specific bonus pay for each operator. Figure 4 illustrates the 
amount of job specific schooling and on-the-job training received by each. 
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Figure 4. Sample Operator Data Sheets. 


The following tables contain the actual case data for the cost of human assets used in this 


research; 


Element 

Annual Unit Cost 
(avq) 

Annual Unit 
Bonuses (avg) 

Budget (Cost) per 
Sample Pd 

Div Officer 

$ 

79,298 

$ 

- 

$ 

39,649 

Div LPO 

$ 

79,298 

$ 

- 

$ 

39,649 

SiqOp 1 

$ 

61,883 

$ 

35,475 

$ 

48,679 

SiqOp 2 

$ 

58,190 

$ 

14,958 

$ 

36,574 

SiqOp 3 

$ 

44,391 

$ 

17,177 

$ 

30,784 

SiqOp 4 

$ 

43,397 

$ 

- 

$ 

21,699 

SiqOp 5 

$ 

41,518 

$ 

- 

$ 

20,759 

ComOpf 

$ 

44,391 


$ 

22,196 

ComOp2 

$ 

43,397 


$ 

21,699 

ComOp3 

$ 

41,518 


$ 

20,759 

Total Human 



$ 

302,446 

Table 8 

USS READIN] 

iSS Human Capital Cost Data. 
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The total amount of days of on-the-job training and job experienee of the human assets 
are shown in Table 9. 


Operator 

Time in Service 
(Days) 

Pre-Depioyment 
Training (Days) 

On-Job Training 
(Days) 

Totais 

Div Officer 

1,825 

15 

292 

2,132 

Div LPO 

4,380 

15 

524 

4,919 

SigOp 1 

3,285 

30 

486 

3,801 

SigOp 2 

2,190 

30 

366 

2,586 

SigOp 3 

1,095 

30 

325 

1,450 

SigOp 4 

730 

30 

219 

979 

SigOp 5 

730 

30 

184 

944 

ComOpf 

2190 

20 

325 

2,535 

ComOp2 

1096 

20 

219 

1,335 

ComOp3 

730 

20 

184 

934 


Table 9. USS READINESS Operator Training Time (Days). 


c. Information Technology Data 

Detailed information was also gathered for the USS READINESS CCOP 
systems and fixed infrastrueture analyzed in this researeh. Cost data, shown below, was 
derived from annual budget estimates. 


Eiement 

Avg Annual Unit 
Costs 

Budget 

(Cost) per Sample 
Pd 

CCOP A 

$ 193,500 

$ 

96,750 

CCOP B 

$ 44,643 

$ 

22,322 

CCOPC 

$ 24,000 

$ 

12,000 

CCOP D 

$ 126,923 

$ 

63,462 

Totai CCOP 


$ 

194,533 

Fixed IT Infrastructure 


$ 

102,500 

TOTAL IT 


$ 

297,033 


Table 10. USS READINESS Systems Cost Data. 


To obtain the time-to-learn of CCOP systems, eaeh was analyzed at the component or 
subsystem-level. The learning-time for each component was then estimated by 
interviewing subject matter experts and deriving how many days it would take the 
average IW operator to learn how to perform the component operation. The table below 
contains the aggregated estimations used for each system. 
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System 

Time to Learn 
(days) 

CCOP A Aggregated Time to Learn = 

4,092 

CCOP B 

1,460 

CCOPC 

514 

CCOP D 

1,825 

Table 11. CCOP System Learning-" 

Tme. 


3. KVA Steps 

The following is a summarized synopsis of the steps in using the KVA method 
onee the required data has been gathered. Two of the USS READINESS ICP 
subproeesses are used to illustrate the computations, subprocesses P4 and P8. Appendix 
A contains the actual analysis. Each step also contains the definitions of variables and 
relevant formulas used. 

a. Step One: Estimate Process Time- to-Learn 

(1) Definitions: 

Time to Learn (11) is the time it would take the average learner to learn how to produce 
a single subprocess output. 

Human Time to Learn (Ilh) is the time it would take the average learner to learn the 
human-specific portions of the subprocess required to produce a single subprocess output. 
In this case factors such as time-in-service, schooling, on the job training, and pre¬ 
deployment training of each operator were used to estimate the human time to learn. 

IT Time to Learn (Ilit) is the time it would take the same average learner to learn how 
to produce the outputs produced by the IT systems in a single subprocess output cycle. In 
this case, subject matter experts in the functional fields of each system were consulted to 
estimate the IT time to learn as exampled in Figure 3. 

% Automation is the percent of a process that is automated by information technology 
(IT) over and above human contribution. 

(2) Description: First, the time it takes an average Information 
Warfare operator to learn one whole subprocess is estimated. This represents the amount 
of knowledge required to produce a single aggregate subprocess output. Time to Learn 
(tn) is estimated by summing the average time spent in training the process (pre¬ 
deployment training), the combined training experience of each operator performing the 
process (Table 9.), and the learning time embedded in each IT system multiplied by the 
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percent automation of the particular subprocess. The time to learn of CCOP systems 
operating in more than one subprocess is divided equally across the subprocess in which 
they operate. 



Sub¬ 

process 

Name 

CCOP 

Assigned 

Process 

Training 

tLH 

(days) 

Other 

Relevant 

tLH 

(days) 

Tot tLH 

(days) 

CCOP 

tLIT 

(days) 

Avg % 
Automat'n 

Tot tuT 
times % 
Automat'n 
(days) 

Tot Ufor 1 
Process 
Output 
(days) 

P4 

Search / 

Collection 

Process 

A,B 

35 

9760 

9795 

896 

46.25% 

414.34 

10,209.34 

P8 

Format 

Data for 
Report 
Generatio 
n 

A 

10 

4804 

4814 

409 

71.67% 

293.27 

5,107.27 


Table 12. P4 and P8 Time to Learn. 


b. Steps Two and Three: Calculate the K Produced by IT and 
Human Assets. Find the Total Kfor Each Subprocess 

(1) Definitions: 

K is the descriptive term chosen for the common units of output estimated by KVA. 

Executions (Ex) are the average number of times a process asset, human or IT, produced 
an individual subprocess output. 

Kh is the common units of output attributed to human-asset contribution. 

Kit is the common units of output attributed to IT-asset contribution. 

Kp is the total common units of output for each subprocess. 

(2) Formulas: 

Total subprocess-asset output: Kassbi = (ExAsset) (tL) 

Total subprocess output: Kp = Kh + Kjt 

Total process output: Ktot = 2(Kp ) 

(3) Description: The number of times an element, human or IT, 
executes a subprocess is tallied and then multiplied by the time to learn of the particular 
subprocess. The resulting number is the amount of output, by asset, used in the execution 
of the subprocess. Once the K for each asset has been calculated, it is summed to reveal 
the total K for each subprocess. 
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Total Output per Subprocess for Sample Period - 
Including Automation 


# of 
each 
asset 
per 
Posi¬ 
tion 

Asset 

# 

executns 
by Asset 
P4 

Total K 

P4 

# 

executns 
by Asset 
P8 

Total K 
P8 

1 

Div Officer 

0.00 

0.00 

0.00 

0.00 

1 

Div LPO 

0.00 

0.00 

0.00 

0.00 

1 

SigOp 1 

135.00 

1378260.68 

0.00 

0.00 

1 

SigOp 2 

144.00 

1470144.72 

0.00 

0.00 

1 

SigOp 3 

162.00 

1653912.81 

0.00 

0.00 

1 

SigOp 4 

152.00 

1551819.43 

0.00 

0.00 

1 

SigOp 5 

158.00 

1613075.46 

0.00 

0.00 

1 

ComOpf 

0.00 

0.00 

78.00 

398367.34 

1 

ComOp2 

0.00 

0.00 

78.00 

398367.34 

1 

ComOpS 

0.00 

0.00 

79.00 

403474.62 


1 

CCOP A 

751.00 

7667213.09 

235.00 

1200209.31 

1 

CCOP B 

92.00 

939259.13 

0.00 

0.00 

1 

CCOPC 

0.00 

0.00 

0.00 

0.00 

1 

CCOP D 

IT Fixed 

0.00 

0.00 

0.00 

0.00 

1 

Infrastructure 

235.00 

2399194.51 

235.00 

1200209.31 



Kp 

18672879.81 


3600627.92 


Table 13. 

P4 and P8 Total K by Asset. 



Subprocess Name 

K|t 

(automation 
& infras) 

Kh 

Ktot 

P 4 

Search/Collection 

10 , 909 , 092.30 

7 , 599 , 933.50 

18 , 509 , 025.80 

P 8 

Format Data for 
Report Generation 

2 , 342 , 721.41 

1 , 171 , 360.71 

3 , 514 , 082.12 


Table 14. P4 and P8 IT/Human Total K. 


c. Steps Four and Five: Derive Proxy Revenue Stream and 
Develop the Value Equation Numerator by Assigning Revenue 
Streams to Subprocesses. 

(1) Definitions: 

Market Comparable Price per Unit is the notional price per unit allocated to the outputs of 
non-profit organizations based on the market price per unit of the comparable outputs of a 
similar commercial organization. 
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% K is percent of the total K produced by an individual subprocess or asset. 

(2) Formulas: 

Proxy Revenue: Rtot = (Total # of Process Outputs) (Market Comp, Price per Unit) 

% of Total K per Subprocess: % Kp = (Kp / Ktot) x 100% 

Subprocess Revenue Allocation: Rp = %Kp x Rtot 


(3) Description: First, utilizing the Market Comparables approach, 
the total number of ICP outputs is multiplied by the average market price-per-unit to 


yield a Proxy Revenue for the USS READINESS ICP. 


Proxy Revenue Assumptions 


Market Comparable Price Per Unit (avg) 

Avg# Reports executed/sample pd 

$ 3,800 

235 

Avg Proxy for Revs - Sample Pd (Rtot) 

$ 893,000 


Table 15. USS READINESS ICP Proxy Revenue Assumption. 


Next, the percent of the total process K produced by each 
subprocess is calculated. 


Subprocess Name 

Kfor IT 

(automation 
& infras) 

Kfor 

Humans 

Total K 

% of Total 

K per sub¬ 
process 

P 4 

Search/Collection io, 909 , 092.30 

7 , 599 , 933.50 

18 , 509 , 025.80 

24 . 5398 % 

P 8 

Format Data for 

Report 

Generation 2 , 342 , 721.41 

1 , 171 , 360.71 

3 , 514 , 082.12 

4 . 6591 % 


52 , 659 , 382.55 

22 , 765 , 206.13 

75 , 424 , 588.68 

100 . 0000 % 


Table 16. P4 and P8 Percent K. 


Revenues can now be assigned to subprocesses, people and IT 
based on their individual %K: 
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Subprocess 

Name 

Kfor 

Humans 

Total K 

% of Total 

K per sub¬ 
process 

Proxy 
Revenue 
Assigned 
to Sub¬ 
process 
($US) 

%of 
Total K 
for 

Human 
per Sub¬ 
process 

Proxy 
Revenue 
Assigned 
to Human 

K ($US) 

P 4 

Search/ 

Collection 7 , 599 , 933.50 

18 , 509 , 025.80 

24 . 5398 % 

$ 219,140 

10 . 0762 % 

$ 89,980 

P 8 

Format Data for 

Report 

Generation i , i7i , 360.71 

3 , 514 , 082.12 

4 . 6591 % 

$ 41,605 

1 . 5530 % 

$ 13,868 


22 , 765 , 206.13 

75 , 424 , 588.68 

100 .0000% 

$ 893,000 

30 . 1827 % 

$ 269,532 


Table 17. P4 and P8 Proxy Revenue Alloeation for Human Contribution. 


Subprocess Name 

Kfor IT 

(automation & 
infras) 

Total K 

%of 
Total K 
for 

CCOP 

A 

Proxy 
Revenue 
Assigned to 
CCOP A 
Process K 
($US) 

%of 
Total K 
for 

CCOP 

B 

Proxy 
Revenue 
Assigned to 
CCOP B 
Process K 
($US) 

P 4 

Search/ 

Collection 

10 , 909 , 092.30 

18 , 509 , 025.80 

10 . 08 % 

$ 89,980 

1 . 23 % 

$ 11,023 

P 8 

Format Data for 
Report 

Generation 

2 , 342 , 721.41 

3 , 514 , 082.12 

1 . 55 % 

$ 13,868 

0 .00% 

$ - 


52 , 659 , 382.55 

75 , 424 , 588.68 

30 . 18 % 

$ 269,532 

3 . 82 % 

$ 34,110 


Table 18. P4 and P8 Proxy Revenue Alloeation for CCOP A & B Contribution. 


Subprocess Name 

%of 
Total K 
for 

CCOP 

C 

Proxy 
Revenue 
Assigned to 
CCOPC 
Process K 
($US) 

%of 

Total 

Kfor 

CCOP 

D 

Proxy Revenue 
Assigned to 
CCOP D 
Process K 
($US) 

% of Total K 
for Fixed IT 
Infrastructure 

Proxy 
Revenue 
Assigned 
to Fixed 
Infras 
Process K 
($US) 

P 4 

Search/ 

Collection 

0 .00% 

$ 

0 .00% 

$ 

3 . 15 % 

$ 28,156 

P 8 

Format Data for 
Report 

Generation 

0 .00% 

$ 

0 .00% 

$ 

1 . 55 % 

$ 13,868 


6 . 52 % 

$ 58,253 

2 . 23 % 

$ 19,906 

27 . 06 % 

$ 241,667 


Table 19. P4 and P8 Proxy Revenue Allocation for CCOP C & D, and IT Fixed 

Infrastructure Contribution. 
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d. Step Six: Develop the Value Equation Denominator by 
Assigning Costs to Subprocesses 

(1) Description; Costs are assigned directly to each subprocess 
based on the assets producing outputs in each. The cost of human and IT assets that are 
assigned to multiple processes are divided evenly throughout those subprocesses. The 
cost of human and IT assets are summed in each subprocess to yield the total cost per 
subprocess (Cp). 


Subprocess Name 

Proxy 
Revenue 
Assigned 
to Sub¬ 
process 
($US) 

Cost 

Assigned 
to Sub¬ 
process 
($US) 

Proxy 
Revenue 
Assigned 
to Human 

K ($US) 

Cost 

Assigned 
to Human 

K 

($US) 

P 4 

Search/Collection 

$ 219,140 

$ 64,642 

$ 89,980 

$ 37 , 276.47 

P 8 

Format Data for 
Report 

Generation 

$ 41,605 

$ 52,252 

$ 13,868 

$ 32 , 326.50 


$ 893,000 

$ 622,607 

$ 269,532 

$ 292,533 


Table 20. P4 and P8 Total Cost Allocation & Human Cost Allocation. 


Subprocess Name 

Proxy 
Revenue 
Assigned 
to CCOP A 
Process K 
($US) 

Cost 

Assigned 
to CCOP 

A 

Process 

K 

($US) 

Proxy 
Revenue 
Assigned 
to CCOP B 
Process K 
($US) 

Cost 

Assigned 
to CCOP B 
Process K 
($US) 

P 4 

Search/ 

Coiiection 

$ 89,980 

$ 9,675 

$ 11,023 

$ 7,441 

P 8 

Format Data for 
Report 

Generation 

$ 13,868 

$ 9,675 

$ - 


Table 21. P4 an 

$ 269,532 

$ 96,750 

$ 34,110 

$ 22,322 

d P8 Cost Allocation for CCOP A & 

B. 
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Proxy 
Revenue 
Assigned 
to CCOP 

C Process 
K 

($US) 

Cost 

Assigned 
to CCOP 

C 

Process 

K 

($US) 

Proxy 
Revenue 
Assigned 
to CCOP D 
Process K 
($US) 

Cost 

Assigned 
to CCOP 

D 

Process 

K 

($US) 

Proxy 
Revenue 
Assigned 
to Fixed 
infras 
Process 

K 

($US) 

Cost 

Assigned 
to Fixed 
infras 
Process 

K 

($US) 

$- 


$- 


$ 28,156 

$ 10,250 

$- 


$- 


$ 13,868 

$ 10,250 

$ 58,253 

$ 12,000 

$ 19,906 

$ 63,462 

$ 241,667 

$ 102,500 


Table 22. P4 and P8 Cost Allocation for CCOP C, D, and Fixed IT Infrastructure. 

e. Steps Seven, Eight and Nine: Calculate the Value Equation 
(ROI) 

(1) Definitions: 

ROK is the Return on Knowledge, a productivity ratio 
ROKA is the Return on Knowledge Assets, a profitability ratio 
ROKI is the Return on Knowledge Investment, the value equation 

(2) Formulas: 

Total Return on Knowledge: ROK = Revenue / Cost 

Subprocess ROK (as percentage): ROKp = (Rp / Cp) x 100% 

Subprocess ROKA: ROKAp = ( Rp — Cp ) / ( %Kp x Rtot ) 

Subprocess ROKI: ROKIp = ( Rp- Cp ) / ( Cp ) 


(2) Description: The revenues and costs assigned to subprocesses, 
people and IT are used to calculate the value equations. 


KVA Metrics 

or Total K 


Subprocess Name 

ROK as Ratio 

ROK as % 

ROKA 

ROKi 

P 4 

Search/Coiiection 

3.39 

339.01% 

70.50% 

239.01% 

P 8 

Format Data for 

Report Generation 

0.80 

79.63% 

-25.59% 

-20.37% 

Metrics for Aggregated 

14.10 

1410.20% 

157.31% 

410.20% 

Table 23. P4andP8 

CVA Metrics. 
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Note: For Fluman and IT ROK, ROKA, and ROKI, the Cost and Revenue of each asset 
is substituted for subprocess cost and revenues in the value equations. 


4. KVA Results for USS REDINESS ICP with Actual Case Data Iputs 


Historical KVA for USS READINESS for Intelligence Collection Process 

Total K Contribution and Human K 


KVA Metrics tor Total K 


KVA Metrics for Human K 



Siibprocess Name 

ROK as 
Ratio 

ROK as % 

ROKA 

ROKI 


Sitbprocess Name 

ROK as 
Ratio 

ROK as % 

ROKA 

ROKI 

PI 

Receive Review 
ReqiiestTaskiiKi 

1.22 

122.11% 

16.10% 

22.11% 

PI 

Receive Review 
ReqirestTaskinq 

0.71 

70.50% 

-41.84% 

-29.50% 

P2 

Deteimlne Op Equip 
Mix 

1.21 

120.89% 

17.28% 

20.89% 

P2 

Determine Op Eqtrtp 
Mix 

0.70 

69.80% 

•43.26% 

•30.20% 

P3 

Load Search 

Fiinc Coverage 

Plan 

0.82 

81.56% 

-22.62% 

-18.44% 

P3 

Load Search 
Func'Covetaqe Plan 

0.96 

95.89% 

-4.29% 

-4.11% 

P4 

Search Collection 

3.39 

339.01% 

70.50% 

239.01% 

P4 

SearchCollection 

2.41 

241.39% 

58.57% 

141.39% 

PS 

Target Data 
Acquisition Capture 

1.47 

147.28% 

32.10% 

47.28% 

PS 

Target Data 
Acquisition Capture 

0.75 

75.31% 

-32.79% 

-24.69% 

PS 

Target Data 
Processinq 

1.37 

136.67% 

26.83% 

36.67% 

P6 

Target Data 
Processinq 

0.83 

82.90% 

-20.63% 

-17.10% 

P7 

Target Data 

Analysis 

1.21 

121.25% 

17.52% 

21.25% 

PI 

Tarqet Data Analysis 

0.63 

62.75% 

-59.35% 

-37.25% 

P9 

Format Data for 
Report Generation 

0.80 

79.63% 

-25.59% 

-20.37% 

P9 

Format Data for 
Report Generation 

0.43 

42.90% 

-133.09% 

•57.10% 

P9 

OC Report 

1.79 

179.19% 

44.19% 

79.19% 

P9 

OC Report 

0.98 

97.82% 

•2.22% 

•2.18% 

P10 

Transmit Report 

0.83 

82.63% 

-21.02% 

-17.37% 

pn 

Transmit Report 

0.45 

44.52% 

•124.62% 

-55.48% 

Metiics for Aaoreoated 

14.10 

1410.20% 

157.31% 

410.20% 


Metrics for Aaareaati 

8.84 

683.79% 

-403.52% 

-116.21% 


Please note tliat tlie flooi foi ROKA is -100% |e.<].. zero letiiiii on knowledge .istjPleose note tlitit tlie floot fot ROKA is -100% (e.g., zero letniii on knowledge assets) 


KVA Metrics for CCOP A K 



Sith-Ptocess 

Name 

ROK .IS R.itio 

ROK as 

ROKA 

ROKI 

PI 

Receive 

Review 

Request 

Tasking 

1.69 

168.54% 

40.67% 

68.54% 

P2 

Determine 

Op Egiiip Mix 

1.67 

166.86% 

40.07% 

66.86% 

P3 

Load Search 
Fitnc/' 

Coverage 

Plan 

1.53 

152.91% 

34.60% 

52.91% 

P4 

Search 

Collection 

9.30 

930.03% 

89.25% 

830.03% 

PS 

Target Data 
Acqirisitioiv 
Capture 

2.90 

290.15% 

65.54% 

190.15% 

P6 

Target Data 
Processing 

3.19 

319.39% 

68.69% 

219.39% 

P7 

Target Data 
Analysis 

1.50 

149.98% 

33.33% 

49.98% 

P» 

Format Data 
fot Report 
Generation 

1,43 

143.34% 

30.24% 

43.34% 

P9 

OC Report 

3.16 

315.88% 

68.34% 

215.88% 

P10 

Transmit 

Report 

1.49 

148.75% 

32.77% 

48.75% 

Metrics for Aaareaated 

27.86 

2785.86% 

503.50% 

1785.86% 


Please note that the floor for ROKA is -100% (e.g.. zero return on knowledge assets) 
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KVA Metrics for CCOP B K 


KVA Metrics for CCOP C K 


Sub-Process 

Name 

ROK as 
Ratio 

ROKas^o 

ROKA 

ROKI 



Sub-Process 

Name 

ROK as 
Ratio 

ROK as % 

ROKA 

ROKI 

PI 

Receive' 

Review 

Request 

Taskitin 






PI 

Receive 

Review 

Request 

Taskinn 





P2 

Deieriniiie 

Op Eduii) Mix 






P2 

Determine 

Op Equip Mix 





P3 

Load Search 
Fiiiio' 

Coverage 

Plan 






P3 

Load Searcli 
Func 

Coverage 

Plan 





P4 

Search 

Collection 

1.48 

148.15% 

32.50% 

48.15% 


P4 

Searcli 

Collection 





PS 

Target Data 

AcquisitioivCa 

ptuie 

1.48 

147.71% 

32.30% 

47.71% 


PS 

Target Data 

AcqirisitionCa 

pture 





P€ 

Target Data 
Processinu 

1.63 

162.59% 

38.50% 

62.59% 


P6 

Target Data 
Process! rur 

4.36 

436.13% 

77.07% 

336.13% 

P7 

Target Data 
Anaivsis 






P7 

Target Data 
Anaivsis 

5.35 

534.76% 

81.30% 

434.76% 

P9 

Format Data 
for Report 
Generation 






PS 

Format Data 
for Report 
Generation 





P9 

OC Report 






P9 

OC Report 





P19 

Transmit 

Report 






P10 

Transmit 

Report 





Metiics for Aqqieqated 

4.58 

458.44% 

103.29% 

158.44% 


Metrics for Aqqreqated 

9.71 

970.88% 

158.37% 

770.88% 


Please note that the flooi for ROKA is -100% (e.rj.. zero letirin on knowledge assets} 


KVA Metrics for CCOP D K 


KVA Metrics for Fixed infrastructure K 


Sub-Process 

Name 

ROK as Ratio 

ROK as % 

ROKA 

ROKI 



Sub-Process 

Name 

ROK as 
Ratio 

ROK as % 

ROKA 

ROKI 

PI 

Receive 

Review 

Reqrrest 

Tasking 






PI 

Receive 

Review 

Regiiest 

Tasking 

1.95 

194.72% 

48.64% 

94.72% 

P2 

Determine 
Op'Eqnii) Mix 






PI 

Determine 

Op Equip Mix 

1.93 

192.78% 

48.13% 

92.78% 

P3 

Load Search 
Frtnc 

Coverage 

Plan 






P3 

Load Search 
Func 

Coverage 

Plan 

1.77 

176.65% 

43.39% 

76.65% 

P4 

Search 

Collection 






P4 

Search’ 

Collection 

2.75 

274.70% 

63.60% 

174.70% 

P5 

Target Data 
Acgiiisition 
Capture 






P5 

Target Data 
Acquisition 
Capture 

2.74 

273.88% 

63.49% 

173.88% 

P6 

Target Data 
Ptocessinq 

0.28 

28.18% 

■254.86% 

-71.82% 


P6 

Target Data 
Processing 

3.01 

301.48% 

66.83% 

201.48% 

P7 

Target Data 
Anaivsis 

0.35 

34.55% 

■189.41% 

■65.45% 


P7 

Target Data 
Anaivsis 

3.70 

369.66% 

72.95% 

269.66% 

P$ 

Format Data 
for Report 
Generation 






P9 

Format Data 
for Report 
Geiieiation 

1.35 

135.30% 

26.09% 

35.30% 

P9 

OC Report 






P9 

OC Report 

2.98 

298.16% 

66.46% 

198.16% 

PfO 

Transmit 

Report 






pn 

Transmit 

Report 

1.40 

140.41% 

28.78% 

40.41% 

Metrics for Aqqreqated 

0.63 

62.73% 

■444.28% 

-137.27% 


Metiics for Aqqreqated 

23.58 

2357.73% 

528.36% 

1357.73% 


Table 24. KVA Results for USS READINESS 


E. ANALYZING THE KVA RESULTS 

KVA provides a unique and insightful means of measuring the eontributions of 
Information Teehnology and human assets to the production of process outputs. By 
analyzing an organization, process or function at the subprocess-level, this methodology 
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allows military leaders and program managers to structure a problem and solution set in 
an empirical manner. The results can then be used to conduct comparative analyses of 
multiple human and IT configurations of similar processes with the final goal of ensuring 
the best possible posturing of people, equipment and the increasingly limited funding- 
dollars within the organization and the Department of Defense. 

While profitability is not the goal of the DoD, and other non-profit organizations, 
it is their goal and responsibility to the individual taxpayer, to ensure that the limited 
resources are managed in a reasonable, accountable and transparent manner. In this case, 
KVA was used to quantify the value added by Cryptologic Carry-On Program systems. 
Information Warfare/Cryptologic operators, and the enabling shipbome infrastructure 
with which they interact. The result is an assessment of the efficiency (productivity) and 
effectiveness (profitability) of human-system performance within the Intelligence 
Collection Process. To obtain a more comprehensive picture of CCOP system 
contribution, multiple iterations of this analysis would have to be run across the Navy¬ 
wide enterprise of intelligence collection platforms. That is not to say though, that 
analyzing a single process yields no value. Looking at the USS READINESS case study 
on its own, it is possible to interpret the KVA analysis results and identify targets of 
transformation within the subprocesses of the Intelligence Collection Process. It is 
possible to look at the individual assets and ask “why?” an asset contributes more or less 
than others, and “how?” the performance of assets impacts the overall output of the 
process. 

1, Case Example Results 

Looking back at the two subprocesses detailed in the case study, P4 and P8, a few 
questions can be asked: 

• What makes P4 such a high performer, with an overall return of 239% 
versus a return of -20.37% for P8? 

• P8, Format Data for Report Generation, only executes once per KL report 
(process output), yet almost a third of all the operators (ComOp 1, 
ComOp2, ComOp3) in this particular case are assigned to this subprocess 
for almost a fifth of the total human cost. 

• P8 is more automated than P4. Could P8 be further automated or could it 
be performed by the other operators in lieu of the three communications 
operators to yield higher efficiency and effectiveness? 
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• P4, Search and Collect, is a knowledge-intensive function requiring a high 
system and human capital asset investment. Each process output requires 
many executions of P4. Could an even higher return be achieved with 
automated search and collection systems or more operators? 

Other observations of this case example could include: 

• Should the amount of knowledge in humans and IT be adjusted? 

• Is a broader range of training required to allow all operators to perform 
more functions? 

• CCOP D is a cost-heavy system but executes very few times, 
comparatively, throughout the sample period. Is CCOP D appropriate for 
this platform and mission? Is there a cheaper alternative to CCOP D? Are 
all the operators appropriately trained in the use of CCOP D? 

These are the kinds of questions CCOP program mangers and other stake-holders 
can begin to ask. The answers to these questions can help CCOP program managers 
allocate funds to new systems or to existing systems for product improvement, or if 
further study deems it so, to cut a system all together from the portfolio. The results can 
also be used to tailor the manning and training requirements of ISR crews deploying with 
CCOP systems. 

Finally, KVA also provides the structured data required to perform various 
methods of risk analysis and performance projections such as Real Options Analysis. 
The combination of KVA historical performance metrics and real options analytic tools 
will enable the CCOP Program Office and the U.S. Navy to estimate and compare the 
future value added of different mixes of humans and systems as well as a range of new 
initiatives for the deployment and employment options of both. 


2. Current Limitations of Knowledge Value Added 

This case study revealed a few limitations to the implementation of KVA to the 
Intelligence Collection Process as modeled in for the USS READINESS, which are 
currently being addressed. They are: 

• With the raw data required for the analysis residing in multiple databases 
of varying classification levels, data-gathering mechanisms that are less 
human-intensive and more automated need to be created to extract the 
required information. 

• Although the ICP in this case study was developed through the use of 
subject matter experts, a standard description and definition of each 
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subprocess should be reaehed through a Intelligenee Community-wide 
consensus of proeess stake-holders. 

• A more detailed researeh should be conducted to analyze the knowledge 
embedded in eaeh IT system to aceurately eapture the benefits resulting 
from the exeeution of partieular system proeesses. 

• The Market Comparables approaeh to valuing the outputs of non-profit 
organizations, although used as a rough baseline to monetize outputs in 
this ease study, requires a more in-depth look at comparable organizations 
utilizing similar processes to produee similar outputs. The ereation of a 
broad database of sueh organizations is eurrently being eondueted to 
benchmark industries by functional groupings and products. 

• Lastly, to provide a more powerful analysis of the ICP, a database of 
eomparable historical KVA information should be created to benehmark 
future work or to provide a broader insight for current work. 
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V. CONCLUSIONS 


Maritime ISR will remain at the core of the Naval Operational Doctrine and will 
continue to be an essential element in improving the speed and effectiveness of naval and 
joint operations. Innovations resulting from the transformation of the DoD’s business 
practices must be refined to meet the unique challenges posed by Intelligence, 
Surveillance, and Reconnaissance programs. It is the responsibility of program managers 
and DoD officials to do their very best to create realistic objectives, clear guidance and an 
effective organization when executing the resources of the United States in defense of the 
nation. Dwight D. Eisenhower once said; 

Every gun that is made, every warship launched, every rocket fired, 
signifies in the final sense a theft from those who hunger and are not fed, 
those who are cold and are not clothed. 

-Dwight D. Eisenhower 

The citizens of the country are entitled to see standards for performance so they can judge 
whether the national objectives are being met in a responsible manner. 

Congress recently criticized the DoD’s acquisition and testing and evaluation of 
Intelligence, Surveillance and Reconnaissance (ISR) systems, stating that there has been 
“inadequate data to compare systems’ capabilities and costs across the spectrum of 
intelligence programs, an imbalance between collection and analysis programs, and an 
intelligence effort that does not reflect an optimal allocation of resources.”36 Section 355 
of the EY2004 Intelligence Authorization Act (P.E. 108-177) required a report from the 
Director of Central Intelligence (DCI) and the Secretary of Defense assessing progress in 
the development of “a comprehensive and uniform analytical capability to assess the 
utility and advisability of various sensor and platform architectures and capabilities for 
the collection of intelligence ... [and] the improvement of coordination between the 
Department [of Defense] and the intelligence community on strategic and budgetary 
planning. ”37 

36 Best, Richard. Intelligence. Surveillance, and Reconnaissance: Issues for Congress . CRS Report 
for Congress (RL 32508). Washington: Congressional Research Service, 22 Feb 2005. p. 2. 

37 Ibid. 
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Assessing the impact and overall value of information technology investments, 
such as ISR systems, has been a continuing challenge, both in business and in the DoD. 
The solution to the valuation of IT systems has been referred to as one of the “holy grails 
of the Information Systems (IS) field,” most notably, at a May 2005 presentation of this 
case study at the DoD’s 2'^'^ Annual Acquisitions Symposium in Monterey, California. But 
even skeptics of the various methodologies proposed for measuring the value of IT 
acquisitions agree that a common framework must be created for understanding, 
evaluating, and in the end, justifying the impact of government investments in 
information technology on the overall successful completion of the national security 
mission of the United States. 

The challenges in determining the value and risk associated with ISR systems 
acquisitions are plenty. In this particular case, the use of Commercial of the Shelf and 
Government off the Shelf (COTS/GOTS) systems and open systems architectures in ISR 
systems provide a level of risk, which must be managed. Technological complexities, the 
reliance on evolving software and systems, shortened acquisitions timelines and funding 
instability all contribute to this risk in Navy ISR systems such as the Cryptologic Carry- 
On Program (CCOP). Although the DoD has instituted rigorous types of testing and 
evaluation (T&E) for all of its programs and projects to mitigate risk, metrics for IT 
systems have lacked the requisite depth for meaningful valuation. Crucial to successful 
T&E is the development of measurable Key Performance Parameters (KPPs) and 
Measures of Effectiveness (MOEs) to provide accurate projections of system 
performance in a variety of operational environments. 

It was the goal of this research to provide the means to extract measures of value 

and effectiveness to the CCOP Program office through the use of the Housel-Kenevsky 

Knowledge Value Added (KVA) Methodology. Applying KVA to the USS 

READINESS Case Study showed that the program managers could build metrics that are 

meaningful and useful in performing sound financial analysis of each system’s 

performance at the process and subprocess level. KVA analysis also identified a new 

category and source of raw data which can provide insights into the relationship of cost 

and value of organizations, processes, and asset investments. This new data allows 

managers and senior decision makers to discuss the “value” of seemingly intangible 
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assets in a defensible, empirical and replicable manner. Lastly, KVA facilitates the 
transformation and continuous process improvement of the DoD’s global intelligence 
mission. Through KVA analysis, the operational value of CCOP systems can be 
measured and managed to ensure a responsible stewardship of the nation’s resources and 
ensure that the soldiers and sailors who use these systems are receiving the right tools 
with the right capabilities required to perform their duties in defense of the nation. 
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VI. RECOMMENDATIONS 


The quest for the holy grail of information teehnology is far from over. An 
aceurate valuation of Cryptologie Cary-On Program systems will eome through an 
organizational transformation and operational transformation as well. 

The Knowledge Value Added (KVA) Methodology, as any other methodology, is 
only as effeetive and meaningful as the data used to apply it. As indieated in the case 
study, data collection challenges remain. It is recommended that processes, both human 
and automated, be created to gather the required performance data in near-real-time. 
Automated logging of system utilization and performance are readily available in many 
business applications. Adapting such mechanisms directly into CCOP systems would 
facilitate the analysis of their performance. Human activity and performance 
mechanisms have also been used in the past, both in business and in the DoD. 

KVA is a robust methodology, when applied throughout an enterprise with the 
appropriate inputs, it is capable of analyzing the data and returning defensible, 
quantifiable results in near-real-time. Although several accounting software packages 
have included KVA analytical capabilities, the research team at the Naval Postgraduate 
School identified the GaussSoft Valuation Software as the most capable software 
platform for conducting the level of analysis required by DoD program managers for 
valuing IT systems and processes. A brief overview of GaussSoft and screen shots of the 
multi-faceted reports produced by the software package are provided in Appendix B. It is 
recommended that this software package be deployed to receive real-time system and 
process inputs from deployed units operating CCOP systems as a proof-of-concept and 
operational test of the software capabilities. 

A true measure of the impact of CCOP systems will require that the methodology 
and data gathering be analyzed and refined over a larger sampling of time. KVA, in its 
current form is ready for deployment; it is recommended that a study be conducted on 
CCOP systems at the Carrier Strike Group (CSG) or Expeditionary Strike Group (ESG) 
level over the course of one deployment to begin establishing performance baselines for 
systems and processes. It is also recommended that a community-wide effort be enacted 
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along with the system developers to determine a universally aeeepted deseription of the 
embedded knowledge and required learning time of each system and process. A more in 
depth and extensive research should be conducted on the Market Comparables Approach 
to include a valuation study of the intelligence products produced by private military 
corporations (PMCs) as well as competitive and business intelligence organizations to 
achieve a common baseline which can be applied directly to DoD intelligence products. 
A KVA baseline should also 

Lastly, it is recommended that an organization be identified to maintain KVA 
databases for CCOP systems. Such organization could be a central repository for system 
performance data as well as a ready source of performance and valuation reports which 
can be tapped quarterly or yearly by program managers wishing to make informed 
acquisition decisions. 
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APPENDIX A. USS READINESS KVA ANALYSIS 


MsMctf Learning lime andAutomrilon DBa -imnlliDiployiiiM■ample Period 

USS READINESS 


Opeidtoi 

Time in 
Service 
(Days) 

Pre- 

Deploy¬ 

ment 

Tiaininq 

(Days) 

OnJol) 

Training 

(Days) 

Totals 

Assigned 

to 

Processe 

s 











Div Officer 

1,825 

15 

292 

2.132 

1,2,9 


CCOP A Aggregated Time to Learn = 

4.092 





Div LPO 

4,380 

15 

524 

4.919 

1,2,9,7 


CCOP B Time to Learn = 


1.460 





SiqOp 1 

3,285 

30 

486 

3.801 

3-7,9 


CCOP C Time to Learn = 


500 





SiqOp 2 

2,190 

30 

366 

2.586 

3-7 


CCOP D Time to Learn = 

1.825 





SidOp 3 

1,095 

30 

325 

1.450 

4-7 










SiqOp 4 

730 

30 

219 

979 

4-6 


Assumptions: 








SiqOp 5 

730 

30 

184 

944 

4-6 









ComOp! 

2190 

20 

325 

2.535 

8,10 









CoinOp2 

1096 

20 

219 

1.335 

8,10 


U Subprocesses over which both IT Infras spread » 

10^ 




Coin0p3 

730 

20 

184 

934 

8,10 


(CCOP System Time to Learn is divided evenly over subprocesses in which they 

operate) 








Sub-Process Name 

CCOP 

Assigned 

Process 

Training 

tLH 

(days) 

Other 

Relevant 

tlH 

(days) 

Tot Ilh 
(days) 

CCOP tLix 

(day^ 

Avg % 
Arrtomat’n 

Tot tLiT 
times % 

Autornat'n 

(days) 

Tot tL for 1 
Process 
Output 
(days) 

PI 

Review ReqiiestTaskinq 

A 

20 

7051 

7071 

409 

25.00% 

102.30 

7.173.30 

PI 

Determine Op Equip Mix 

A 

10 

7051 

7061 

409 

10.00% 

40.92 

7,101.92 



P3 

Input Search Function Coveraqe Plan 

A 

35 

6387 

6422 

409 

21.00% 

85.93 

6507.93 



P4 

Search Collection Process 

A,B 

35 

9760 

9795 

896 

36.25% 

324.75 

10,119.75 



|P5 

Tarqet Data Acquisition Capture 

A,B 

16 

9760 

9776 

896 

35.00% 

313.55 

10,089.55 



PS 

Tarqet Data Processinq 

A.B.C.D 

340 

9760 

10100 

2058 

48.89% 

1.006.34 

11,106.34 



P7 

Tarqet Data Analysis 

A.C.D 

50 

12756 

12806 

1572 

51.67% 

812.10 

13,618.10 



P9 

Format Data for Report Generation 

A 

10 

4804 

4814 

409 

41.67% 

170.51 

4584.51 



P9 

OC Report 

A 

30 

10852 

10882 

409 

25.00% 

102.30 

10.984.30 



PfO 

Transmit Report 

A 

14 

4804 

4818 

409 

86.67% 

354.65 

5,172.65 











83545 



3.313.36 

86.858.36 





Sitbprocess Name 

Total tLiT 
times % 
Arrtomat'n 
(days) 

Total tLH 
(rlays) 

Total tL for 

1 Process 

Executns 

(days) 

Range of Values for 
Non-Routine tL 

ASSUMPTIONS 

PI 

Review RequestTaskiiuj 

102.30 

7.071.00 

7.173.30 

Upper I Lower 


Sample Pd 

Prior Pd 


P2 

Determine Op Equip Mix 

40.92 

7.061.00 

7.101.92 


Avg # Reports during sample period 

235 


Range of Vahres 

PZ 

Input Search 

85.93 

6.422.00 

6.507.93 

10.00% 

-10.00% 

Lenath of sample period as % 

100,00% 

0.00% 

Upper 

Lower 

P4 

Search Collection Process 

324.75 

9.795,00 

10,119,75 

20,00% 

-20.00% 

Avq # Reports executed sample pd 

235 


20.00% 

-20.00*4 

PS 

Tarqet Data 

313.55 

9.776.00 

10,089.55 

20.00% 

-20.00% 





P6 

Tarqet Data Processing 

1,006.34 

10.100.00 

11.106.34 

10.00% 

-10.00% 





P7 

Target Data Analysis 

812.10 

12.806.00 

13.618.10 

10,00% 

-10.00% #Ships in this period = 

1 




PZ 

Format Data for Report 

170.51 

4.814.00 

4,984.51 

15.00% 

-15.00%|#SigOps/Category/Ship = 

1 




P9 

OC Report 

102.30 

10.882,00 

10,984.30 

All otlieitt cannot vary. #SiciOp Cateqories/Ship = 

4 




PIC 

Transmit Report 

354.65 

4,818.00 

5,172.65 

since |)focesses are # CCOPs/Category/Ship = 

1 








3,313.36 

83.545.00 

86.858.36 

roirtine. 

#CCOP Categories/Ship = 

4 











# Command Teams/Ship = 

1 

























# units of time = 

365 
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1 

Knowledge in 

Use for Sample Period - Including Automation 




Asset 


u 

execiitns 
by Asset 

PI 

Total K 

PI 

# execiitns 
by Asset 

P2 

Total K 

PI 

# execiitns 
by Asset 

P3 

Total K 
P3 

n 

execiitns 
by Asset 
P4 

Total K 

P4 

) 

Div Officer 


92.00 

659943.60 

92.00 

653376.64 

0.00 

0.00 

0.00 

0.00 

1 

Div LPO 


100.00 

717330.00 

100.00 

710192.00 

0.00 

0.00 

0.00 

0.00 

1 

SigOp 1 


0.00 

0.00 

0.00 

0.00 

100.00 

650793.20 

135.00 

1366166.48 

) 

SiaOp 2 


0.00 

0.00 

0.00 

0.00 

92.00 

598729.74 

144.00 

1457244.24 

1 

SiqOp 3 


0.00 

0.00 

0.00 

0.00 

0.00 

0.00 

162.00 

1639399.77 


SiqOp 4 


0.00 

0.00 

0.00 

0.00 

0.00 

0.00 

152.00 

1538202.25 

1 

SiqOp 5 


0.00 

0.00 

0.00 

0.00 

0.00 

0.00 

158.00 

1598920.76 

1 

ComOpI 


0.00 

0.00 

0.00 

0.00 

0.00 

0.00 

0.00 

0.00 

1 

ContOp2 


0.00 

0.00 

0.00 

0.00 

0.00 

0.00 

0.00 

0.00 


ComO|)3 


0.00 

0.00 

0.00 

0.00 

0.00 

0.00 

0.00 

0.00 

) 











1 











1 

CCOP A 


192.00 

1377273.60 

192.00 

1363568.64 

192.00 

1249522.94 

751.00 

7599933.50 

i 

CCOP B 


0.00 

0.00 

0.00 

0.00 

0.00 

0.00 

92.00 

931017.15 

) 

CCOP C 


0.00 

0.00 

0.00 

0.00 

0.00 

0.00 

0.00 

0.00 


CCOP D 


0.00 

0.00 

0.00 

0.00 

0.00 

0.00 

0.00 

0.00 

1 

IT Fixed 


235.00 

1685725.50 

235.00 

1668951.20 

235.00 

1529364.02 

235.00 

2378141.64 

) 

liifiestiuctiire 


4440272.70 


4396088.48 


4028409.91 


18509025.80 


T I- 1 r 


# execiitiis by 
Asset P5 


Totiil K 
PS 


# 

execiitns 

by Asset Totiil K 
P6 P6 


tf execiitns 
by Asset Totiil K 

P7 P7 


Tf execiitns 
by Asset Totiil K 
P8 PS 


execiitns 

by Asset Total K 
P9 P9 


# 

execiitns 

by Asset Total K 
P10 P10 


0.00 

0.00 


0.00 

0.00 


0.00 

0.00 


0.00 


0.00 


24.00 326834.34 


0.00 

0.00 


0.00 

0.00 


78.00 856775.40 

78.00 856775.40 


0.00 

0.00 


0.00 

0.00 


56.00 

565014.99 

56.00 

621954.79 

28.00 

381306.73 

0.00 

0.00 

79.00 

867759.70 

0,00 

0.00 

63.00 

635641.86 

63.00 

699699.13 

30.00 

408542.92 

0.00 

0.00 

0.00 

0.00 

0.00 

0.00 

45.00 

454029.90 

45.00 

499785.10 

8.00 

108944.78 

0.00 

0.00 

0.00 

0.00 

0.00 

0.00 

35.00 

353134.37 

35.00 

388721.74 

0.00 

0.00 

0.00 

0.00 

0.00 

0.00 

0.00 

0.00 

36.00 

363223.92 

36.00 

399828.08 

0.00 

0.00 

0.00 

0.00 

0.00 

0.00 

0.00 

0.00 

0.00 

0.00 

0.00 

0.00 

0.00 

0.00 

78.00 

388792.06 

0.00 

0.00 

78.00 

403466.98 

0.00 

0.00 

0.00 

0.00 

0.00 

0.00 

78.00 

388792.06 

0.00 

0.00 

78,00 

403466,98 

0.00 

0.00 

0.00 

0.00 

0.00 

0.00 

79.00 

393776.58 

0.00 

0.00 

79,00 

408639.64 

























235.00 

2371045.03 

235.00 

2609988.83 

90.00 

1225628.77 

235.00 

1171360.71 

235.00 

2581310.50 

235,00 

1215573.61 

92.00 

928238.91 

92.00 

1021782.86 

0.00 

0.00 

0.00 

0.00 

0.00 

0.00 

0.00 

0.00 

0.00 

0.00 

199.00 

2210160.76 

199.00 

2710001.38 

0.00 

0.00 

0.00 

0.00 

0,00 

0.00 

0.00 

0.00 

68.00 

755230.81 

68.00 

926030.62 

0.00 

0.00 

0.00 

0.00 

0,00 

0.00 

235.00 

2371045.03 

235.00 

2609988.83 

235.00 

3200252.89 

235.00 

1171360.71 

235.00 

2581310.50 

235.00 

1215573.61 


8041374.01 


11817140.93 


9287542.42 


3514082.12 


7743931.50 


3646720.82 
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Historical KVA for USS READINESS for Intelligence Coilection Process 

Total K Contribution and Human K 


Assi 
giie 
<1 to 
Pio 

ces 

ses 

Asset 

Avg Annual 
Unit Costs 

Avg Annual 
Unit Bonus 

Budget 
{Cost) per 

Sample P<l 

Cost per 
Operator 
per Sub- 
process 


Proxy Revenue & Cost Assumptions 


1,2.9 

Div Officei 

$ 79,298 

j 

$ 39.649 

$13,216.33 


Maiket Comparable Price Per Unit (avg) 

$ 3.800 


1,2.9 

DIvLPO 

$ 79,298 

I 

$ 39.649 

$9,912.25 


Avg? Reports executed sample pd 

235 

3-7 .S 

$i(|0p 1 

$ 61,883 

$ 35,475 

$ 48.679 

$8,113.17 


Proxy for Revs - Sample Pd « 

$ 893,000 

3-7 

SiqOp 2 

$ 58,190 

$ 14,958 

$ 36.574 

$7,314.80 


Avg cost for IT Fixed Infrastructure (annual) - 

$ 205.000 


4-7 

SiqOp 3 

$ 44,391 

$ 17,177 

$ 30.784 

$7,696.00 


All other fixed costs (aninial) = 


$ 


4-6 

Si(|Op 4 

$ 43,397 

$ 

$ 21.699 

$7,232.83 




4-6 

SiqOp 5 

$ 41,518 

j 

$ 20,759 

$6,919.67 

Length of Sample Pd as % of Year - 

50.00*0 


! 8.10 

ComOpI 

$ 44,391 


$ 22.196 

$11.097.75 






8.10 

ComOp2 

$ 43,397 


$ 21.699 

$10,849.25 






I 


8,10 

ComOp3 

$ 41,518 


$ 20,759 

$10,379.50 









Total Human 



$ 302.446 









i 1-10 

CCOP A 

$ 193,500 


$ 96.750 









' 4-6 

CCOP B 

$ 44,643 


$ 22.322 









) 6,7 

CCOP C 

$ 24,000 


$ 12,000 









) 6,7 
) 

CCOP D 

$ 126,923 


$ 63,462 









Total IT 



$ 194.533 









1-10 

Fixed IT liifrastnicture 


$ 102,500 









! 1-10 

Ottiei Fixed Costs 












) 

GRAND TOTALS 



$ 599,479 










Historical KVA for USS READINESS for Intelligence Collection Process 

Total K Contribution and Human K 


Subprocess Name 

Kfor IT 

(Automation S 
infras) 

K for Humans 

Total K 

% of Total 

K per sub- 
process 

Proxy 
Revenue 
Assigned 
to Sub- 
process 
($US) 

Cost 

Assigned 
to Sub¬ 
process 
($US) 

% of 
Total K 
for 

Human 
per Sub- 
process 

Proxy 
Revenue 
Assigned to 
Human K 
($US) 

Cost 

Assigned to 
Human K 
($US| 

PI 

Receive Review 
ReqiiestTaskinq 

3,062.999.10 

1,377,273,60 

4,440,272.70 

5.8870% 

$ 52,571 

$ 

43,054 

1.8260% 

$ 

16306 

$23,128.58 

P2 

Determine Op Eguip 
Mix 

3,032,519.84 

1,363,568,64 

4,396,088.48 

5.8285% 

$ 52,048 

$ 

43,054 

1.8079% 

$ 

16,144 

$23,128.58 

P3 

Load Search 
FunC'Covetage 

Plan 

2,778.886.96 

1,249,522.94 

4,028,409.91 

5.3410% 

$ 47,695 

$ 

58,482 

1.6567% 

$ 

14,794 

$15,427.97 

P4 

Search Collection 

10,909,092.30 

7,599,933.50 

18,509,025.80 

24.5398% 

$ 219,140 

$ 

64,642 

10.0762% 

$ 

89,980 

$37,276.47 

P5 

Target Data 
Acquisition Capture 

5,670,328.97 

2,371,045.03 

8,041.374.01 

10.6615% 

$ 95,207 

$ 

64,642 

3.1436% 

$ 

28.072 

$37,276.47 

P6 

Target Data 
Processing 

9,207,152.10 

2,609,988.83 

11,817,140.93 

15.6675% 

$ 139.911 

$ 

102,373 

3.4604% 

$ 

30,901 

$37,276.47 

P7 

Target Data 

Analysis 

8,061.913.65 

1,225,628.77 

9.287,542.42 

12.3137% 

$ 109,961 

$ 

90,692 

1.6250% 

$ 

14,511 

$23,123.97 

P8 

Format Data for 
Report Generation 

2,342,721.41 

1,171,360.71 

3,514.082.12 

4.6591% 

$ 41,605 

$ 

52,252 

1.5530% 

$ 

13,868 

$32,326.50 

P9 

OC Report 

5,162,621.00 

2,581,310.50 

7,743,931.50 

10.2671% 

$ 91,685 

$ 

51.167 

3.4224% 

$ 

30,562 

$31,241.75 

pro 

Transmit Report 

2,431.147.21 

1,215,573.61 

3,646.720.82 

4.8349% 

$ 43,176 

$ 

52,252 

1.6116% 

$ 

14,392 

$32,326.50 



52,659.382.55 

22,765,206.13 

75,424.588.68 

100.0000% 

$ 893,000 

$ 

622,607 

30.1827% 

$ 

269,532 

$ 292,533 
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Historical KVA for USS READINESS for Intelligence Collection Process 

Total K Contribution and Human K 


KVA Metrics for Total K 


KVA Metrics for Human K 



Suhpiocess Name 

ROK as 
Ratio 

ROK as % 

ROKA 

ROKI 


Subpiocess Name 

ROK as 
Ratio 

ROK as % 

ROKA 

ROKI 

PI 

Receive Review 
RequestTaskina 

1.22 

122.11% 

18.10% 

22.11% 

PI 

Receive Review 
RequestTaskinq 

0.71 

70.50% 

-41.84% 

-29.50% 

P2 

Determine Op Equip 
Mix 

1.21 

120.89% 

17.28% 

20.89% 

P2 

Determine Op Equip 
Mix 

0.70 

69.80% 

-43.26% 

-30.20% 

P3 

Load Search 
FuncCoverage 

Plan 

0.82 

81.56% 

-22.62% 

-18.44% 

P3 

Load Search 
Fimc'Coveiaqe Plan 

0.96 

95.89% 

-4.29% 

-4.11% 

P4 

Search’Collection 

3.39 

339.01% 

70.50% 

239.01% 

P4 

Search'Collection 

2.41 

241.39% 

58.57% 

141.39% 

P5 

Target Data 
Acquisition Captine 

1.47 

147.28% 

32.10% 

47.28% 

PS 

Target Data 
Acquisition Capture 

0.75 

75.31% 

-32.79% 

-24.69% 


Target Data 
Piocessinq 

1.37 

136.67% 

26.83% 

36.67% 

P6 

Target Data 
Pfocessinq 

0.83 

82.90% 

-20.63% 

-17.10% 

P7 

Target Data 

Analysis 

1.21 

121.25% 

17.52% 

21.25% 

P7 

Tarqet Data Analysis 

0,63 

62.75% 

-59.35% 

-37.25% 

PS 

Format Data for 
Repoit Generation 

0,80 

79.63% 

-25.59% 

-20.37% 

PS 

Format Data for 
Report Generation 

0,43 

42.90% 

-133,09% 

-57.10% 

P9 

OC Report 

1.79 

179.19% 

44.19% 

79.19% 

P9 

QC Report 

0.98 

97.82% 

-2.22% 

-2.18% 

PfO 

Transmit Report 

0.83 

82.63% 

-21.02% 

-17.37% 

PfO 

Transmit Report 

0.45 

44.52% 

-124.62% 

-55,48% 

Metiics foi Aaareqatetl 

14.10 

1410.20% 

157,31% 

410.20% 


Metrics for Agareqatr 

8.84 

883.79% 

-403.52% 

-116.21% 


Pleose note tiuit the floor for ROKA is -100°o (e.g.. zero return on knowledge assjPleose note tiuit the floor for ROKA is -100°o (e.g.. zero return on knowledge 
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Historical KVA for USS READINESS for Intelligence Collection Process 

IT Conti ihiitloii 


Pion Revenue & Cos! Assumptions 


CCOP A 

K 

CCOPB 

K 

CCOPC 

K 

CCOPD 

K 

Filed Infras 
K 

Total IT 

K 

Market Comparable Price Per Unit (avg) 

$ 3.800 

Pt 

1,377.273.60 




1,685,725.50 

3,062,333.10 

Avgt Reports eiecutedfsample pd 

235 

PP 

1,363,568.64 




1,668,351.20 

3,032,513.84 

Avg Proig for Revs - Sample Pd s 

$ 893.000 

PS 

1,243.522.34 




1,523,364.02 

2,778,886.36 

Avg cost for IT Fried Infrastructure (annual) : 

$ 205.000 

P4 

7.533,333.50 

331,017.15 



2.378,141.64 

10,303,032.30 

All other filed costs (annual): 

♦ 

PS 

2,371,045.03 

328,238.31 



2,371,045.03 

5,670,328.37 




PS 

2,603.388.83 

1,021,782.86 

2,210,160.76 

755,230.81 

2,603,388.83 

3,207,152.10 

Length of Sample Pd as X of Year s 


50.00X 

Pf 

1,225.628.77 


2.710,001.38 

326,030.62 

3,200,252.83 

8,061,313.65 






PS 

1,171,360.71 




1,171,360.71 

2,342,721.41 






PS 

2,581,310.50 




2,581,310.50 

5,162,621.00 






PtS 

1,215,573.61 




1,215,573.61 

2,431,147.21 







22,765,206.13 

2,681,038.32 

4,320,162.14 

1,681,261.43 

20,411,713.33 

52,658,882.55 


Subproeess Name 

K for IT 

iafras) 

K for 

Humans 

Total K 

X of Total K 
for CCOP A 

Proig 
Revenue 
Assigned 
to CCOP 

A Process 

K 

(*US) 

Cost 
Assigned 
to CCOP 

A 

Process 

K 

(*US) 

X of Total K 
for CCOP B 

Proig 
Revenue 
Assigned 
to CCOP 

B Process 

K 

(»US) 

Cost 
Assigned 
to CCOP B 

Process K 
(»US) 

Pt 

Keceivel 

Review 

Request^ 

Tasking 

3,062,333.10 

1.377.273.60 

4.440.272.70 

1.83!t 

t 

16,306 

t 

3,675 

0.00% 

J 


pp 

Determine 
OpfEquip Mil 

3,032,513.84 

1.363,568.64 

4,336,088.48 

1.81i 

t 

16,144 

s 

3,675 

0.00% 

t 


PS 

Load Search 

Funcf 

Coverage 

2.T78.886.36 

1,243,522.34 

4,028.403.31 

1.66% 

t 

14,734 

t 

3,675 

0.00% 

t 


P4 

Search/ 

Collection 

10.303.032,30 

7,533.333.50 

18,503,025.80 

10.08% 

t 

83,380 

( 

3.675 

1.23% 

t 

11,023 

% 

7.441 

PS 

Target Data 
Acquisition/ 
Capture 

5.670,328.37 

2,371.045.03 

8,041.374.01 

3.14% 

t 

28,072 

t 

3.675 

1.23% 

t 

10,330 

% 

7,441 

PS 

Target Data 
Processing 

3,207,152.10 

2,603,388.83 

11,817.140.33 

3.46% 

t 

30,301 

t 

3.675 

1.35% 

t 

12,038 

t 

7,441 

PT 

Target Data 
Anaigsis 

8.061.313.65 

1,225,628.77 

3.287.542.42 

1.62% 

t 

14,511 

i 

3.675 

0.00% 

* 


PS 

Format Data 
for Report 
Generation 

2.342,721.41 

1,171,360.71 

3,514,082.12 

1.55% 

t 

13,868 

$ 

3,675 

0.00% 

J 


PS 

QC Report 

5.162.621.00 

2,581,310.50 

7,743,331.50 

3.42% 

t 

30,562 

t 

3,675 

0.00% 

1 


PtS 

Transmit 

Report 

2.431,147.21 

1,215,573.61 

3,646.720.82 

1.61% 

t 

14,332 

t 

3,675 

0.00% 

t 



52.653,382.55 

22,765.206.13 

75,424,588.68 

30.18% 

t 

263,532 

t 

36,750 

3.82% 

t 

34,110 

% 

22.322 
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Siibprocess Name 

% of Total K 
for CCOP C 

Proxy 
Revenue 
Assigned to 
CCOP C 
Process K 
($US) 

Cost 

Assigned to 
CCOP C 
Process K 
($US) 

% ofTotal K 
for CCOP D 

Proxy 
Revenue 
Assigned to 
CCOP D 
Process K 
i$US) 

Cost 

Assigned 
to CCOP 
D 

Process 

K 

i$US) 

% of Total 
Kfor 
Fixed IT 
Infrastriict 

lire 

Proxy 
Revenue 
Assigned 
to Fixed 
Infras 
Process K 
(tUS) 

Cost 

Assigned to 
Fixed 
Infras 
Process K 
{$US) 

PI 

Receive 

Review 

Request 

Task! II (| 


J 



$ 


0.022350 

5 

19,958 

$ 

10,250 

P2 

Determine 

0|) Equip Mix 


% 



$ 


0.022127 

$ 

19,760 

$ 

10,250 

P3 

Load Search 
Fimc' 

Coveiaqe 


% 



$ 


0.020277 

5 

18,107 

$ 

10,250 

P4 

Search. 

Collection 

0.00% 

$ 


0.00% 

$ 


3.15% 

$ 

28,156 

$ 

10,250 

P5 

Target Data 
Acquisition 
Capture 

0.00% 

$ 


0.00% 

$ 


3.14% 

$ 

28,072 

$ 

10,250 

P6 

Target Data 
Processing 

2.93% 

$ 

26,168 

$ 6,000 

1.00% 

$ 

8,942 

$ 31,731 

3.46% 

5 

30,901 

$ 

10,250 

P7 

Target Data 
Analysis 

3.59% 

$ 

32,085 

$ 6,000 

1.23% 

$ 

10,964 

$ 31,731 

4.24% 

$ 

37,890 

$ 

10,250 

P9 

Format Data 
for Report 
Generation 

0.00% 

$ 


0.00% 

$ 


1.55% 

$ 

13,868 

$ 

10,250 

P9 

OC Report 

0.00% 

$ 


0.00% 

$ 


3.42% 

$ 

30,562 

$ 

10,250 

P10 

Transmit 

Report 

0.00% 

$ 


0.00% 

$ 


1.61% 

5 

14,392 

$ 

10,250 


6.52% 

$ 

58,253 

$ 12,000 

2.23% 

$ 

19,906 

$ 63,462 

27.06% 

$ 

241,667 

$ 

102,500 


KVA Metrics for CCOP A K 



Sill)-Process 
Name 

ROK as Ratio 

ROK as % 

ROKA 

ROKI 

PI 

Receive 

Review 

Request 

Tasking 

1.69 

168.54% 

40.67% 

68.54% 

P2 

Determine 

Op Equip Mix 

1.67 

166.86% 

40.07% 

66.86% 

P3 

Load Search 

Fnnc 

Coverage 

Plan 

1.53 

152,91% 

34.60% 

52.91% 

P4 

Search 

Collection 

9.30 

930.03% 

89.25% 

830.03% 

P5 

Taiget Data 
Acquisition 
Capture 

2.90 

290.15% 

65.54% 

190.15% 

pe 

Target Data 
Processing 

3,19 

319.39% 

68.69% 

219.39% 

P7 

Target Data 
Analysis 

1.50 

149.98% 

33.33% 

49.98% 

P9 

Foimat Data 
for Report 
Generation 

1.43 

143.34% 

30.24% 

43.34% 

P9 

QC Report 

3.16 

315,88% 

68.34% 

215.88% 

Pie 

Transmit 

Repoit 

1.49 

148.75% 

32.77% 

48.75% 

Metrics for Aaareaated 

27.86 

2785.86% 

503.50% 

1785.86% 


Please note that the floor for ROKA is -100% (e.g., zero letiiin on knowledge assets) 
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KVA Metrics for CCOP C K 


Sub-Process 

Name 

ROK as 
Ratio 

ROK as % 

ROKA 

ROKI 



Sub-Process 

Name 

ROK as 
Ratio 

ROK as % 

ROKA 

ROKI 

PI 

Receive' 

Review 

Request 

Taskinn 






PI 

Receive 

Review 

Request 

Taskino 





P2 

Deieriniiie 

Op Eduii) Mix 






P2 

Determine 

Op Equip Mix 





P3 

Load Search 
Fiiiic' 

Coverage 

Plan 






P3 

Load Search 
Fiinc 

Coverage 

Plan 





P‘4 

Search 

Collection 

1.48 

148.15% 

32.50% 

48.15% 


P4 

Search 

Collection 





PS 

Target Data 

AcqiiisitionCa 

pture 

1.48 

147.71% 

32.30% 

47.71% 


PS 

Target Data 

Acqirisition/Ca 

pture 





P€ 

Target Data 
Process! iKi 

1.63 

162.59% 

38.50% 

62.59% 


PS 

Target Data 
Processinq 

4.36 

436.13% 

77.07% 

336.13% 

P7 

Target Data 
Anaivsis 






P7 

Target Data 
Anaivsis 

5.35 

534.76% 

81.30% 

434.76% 

P9 

Format Data 
for Report 
Generation 






PS 

Format Data 
for Report 
Generation 





P9 

OC Report 






P9 

OC Report 





PfO 

Transmit 

Report 






P10 

Transmit 

Report 





Metrics for Aqareqated 

4.58 

458.44% 

103.29% 

158.44% 


Metrics for Aqqreqated 

9.71 

970.88% 

158.37% 

770.88% 


Please note that the flooi for ROKA is -100% (e.g.. zero letiiin on knowledge assets} 
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KVA Metrics for Fixed infrastructure K 


Sub-Process 

Name 

ROK as Ratio 

ROK as % 

ROKA 

ROKI 



Sub-Process 

Name 

ROK as 
Ratio 

ROK as % 

ROKA 

ROKI 

PI 

Receive 

Review 

Reqrrest 

Tasking 






PI 

Receive 

Review 

Reguest 

Tasking 

1.95 

194.72% 

48.64% 

94.72% 

P2 

Determine 
Op'Eqiiii) Mix 






PI 

Determine 

Op Equip Mix 

1.93 

192.78% 

48.13% 

92,78% 

P3 

Load Search 
Frinc 

Coverage 

Plan 






P3 

Load Search 
Func 

Coverage 

Plan 

1,77 

176.65% 

43.39% 

76,65% 

P4 

Search 

Collection 






P4 

Search 

Collection 

2.75 

274,70% 

63.60% 

174,70% 

P5 

Target Data 
Acgiiisitioi) 
Capture 






P5 

Target Data 
Acquisition: 
Capture 

2,74 

273.88% 

63.49% 

173,88% 

PS 

Target Data 
Processing 

0.28 

28.18% 

■254.86% 

-71.82% 


P6 

Target Data 
Processing 

3.01 

301.48% 

66.83% 

201.48% 

P7 

Target Data 
Anaivsis 

0.35 

34.55% 

■189.41% 

■65.45% 


P7 

Target Data 
Anaivsis 

3.70 

369.86% 

72.95% 

269,66% 

PS 

Format Data 
for Report 
Generation 






PS 

Format Data 
for Report 
Geiietation 

1.35 

135.30% 

26,09% 

35,30% 

P9 

OC Report 






P9 

OC Report 

2.98 

298,16% 

66.46% 

198.16% 

PfO 

Transmit 

Report 






pn 

Transmit 

Report 

1,40 

140,41% 

28,78% 

40,41% 

Metrics for Aqqreqated 

0.63 

62.73% 

■444.28% 

-137.27% 


Metrics for Aqqreqated 

23.58 

2357.73% 

528.36% 

1357,73% 
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APPENDIX B. GAUSSSOFT OVERVIEW 


[GAUSS Overview provided eourtesy of GaussSoft, Ine. <http;www.gaussoft.oom>] 

GAUSS is a line of software ereated by GaussSoft, Ine., a privately held US 
eorporation founded in 1993, with headquarters in San Jose, California and an extended 
presenee with offiees and partners in NorthAmeriea, Europe and Latin Ameriea. 

GaussSoft delivers sealable Enterprise Performanee Intelligenee solutions. 
GaussSoft produets enable large and medium-sized eompanies to eontrol and reduee the 
eost of enterprise operations, inerease profitability and improve organizational 
produetivity by providing flexibility, sealability and ease of use. 

GaussSoft’s solutions are built on an integrated suite of high performanee 
produets for Profit and Cost Analysis, Multidimensional Query, and Aetivity Reporting 
that are sealable, funetion-rieh, and easy to use. 

GaussSoft has installed performanee intelligenee solutions in over 200 enterprise 
and eonsulting eompanies all around the world, ineluding teleeommunieation, banking, 
manufaeturing and agribusiness firms and government organizations. They have been 
implemented in eustomer premises by leading eonsulting firms ineluding Deloitte, 
KPMG and Priee. 

GaussSoft suite ineludes: 

Gauss - Profit and Cost Allocation Engine: This strategie deeision-making and 
analysis solution enables eompanies to know whieh produets, serviees, and eustomers are 
making profits and whieh aren't. Using different value and eosting methodologies this 
solution helps reduee and eontrol the eost of enterprise operations, inerease profitability 
and improve organizational produetivity. 

Gauss - KVA: Knowledge Value Added (KVA) is a methodology that allows any 
organization to ealeulate the eeonomie performanee of eore proeesses by providing an 
objeetive way to alloeate revenue to the proeesses at any level within the organization. 
Knowing how mueh revenue eorporate knowledge is produeing, allows organizations to 
dramatieally improve their effeetiveness and effieieney. 
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Gauss - Planning; This enterprise collaborative solution allows thousands of 
users to perform corporate enterprise planning, including financial planning, budgeting 
and forecasting up to 10 times faster. When used with Gauss Profit and Gauss KVA, an 
organization can create plans optimized for profitability and value. 

Gauss - Radial Viewer; This is a Business Intelligence (BI) front-end with 
graphical interaction. This tool enables all End Users to create their own queries and 
professional looking reports from scratch -in seconds-. 

Figures 5-7 are graphical outputs of GaussSoft products. 



a ^ tj an® 


DomirtBrrt4 


I OQ » M I I acili)' 

Dominants 



Profit and Loss (Account detail) / SP01 SP01 / Lines; ^ 



Total 108,707.54 No.Units: 24.00 Cost p. unit: 4,529.48 Terminal: T 


Figure 5. GaussSoft Accumulator View for KVA Case Study. 
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Figure 6. 


GaussSoft Radial Viewer Report Design Sereen. 



S Widths 1 0 Auto V] 


Process type * 

Process 

KVA 

Revenue 

Cost 

Profit 

ROI 

Human 

SP01 SubprocessOt 

1 

14 gfii 

108708 

-93.746 

-86 


SP02 Subprocess02 

1 

7,481 

32,489 

-25,009 

■77 


SPO3SP03 

2 

26.102 

56,866 

-30.684 

•54 


SPO4SP04 

52 

589,093 

40,335 

548,758 

1,360 

Total Human 

SB 

637.717 

238.398 

399.319 

168 

1 

ITP01 PROC 01 

16 

202 598 

300 000 

-97 402 

-32 


FTPAPROCA 

23 

264,936 

193,500 

71,436 

37 


fTPBPROCB 

4 

46,909 

44.643 

2.266 

5 


ITPCPROCC 

3 

29,486 

24.000 

5,486 

23 


nPO PROC D 

6 

72.374 

126.923 

-54.549 

-43 

Total It 

54 

615.»4 

588.008 

-72.752 

-11 

Hot defined 

SPO5SP0S 

g 

102,234 

31,457 

70,777 

225 


SPO6SP06 

42 

479 223 

30199 

449 024 

1 4H7 


SPOT SPOT 

28 

319,482 

17,850 

301,631 

1.690 


SPO8SP08 

6 

63,896 

16.704 

47.192 

283 


spoospog 

17 

191.689 

26.943 

164,746 

611 


SP10 SP10 

8 

89.455 

46.423 

43.032 

93 

Total Not defined 

110 

U45.0T8 

188.577 

1.076 402 

635 

lOTAL 

_220_ 

_zsimiDa_ 

_USLQii_ 

_Liozofia_ 

_ m. 


Figure 7. GaussSoft Radial Viewer Sample Report 
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